-- Logs begin at Tue 2022-10-04 07:54:14 UTC, end at Tue 2022-10-04 09:02:59 UTC. -- Oct 04 08:26:23.837436 dv-cinder-860209-1-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 04 08:26:23.844979 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 08:26:23 2022] *** Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: nodename: dv-cinder-860209-1-iscsi Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: machine: x86_64 Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: clock source: unix Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: pcre jit disabled Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: detected number of CPU cores: 4 Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: current working directory: / Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: detected binary path: /usr/bin/uwsgi-core Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: your processes number limit is 47680 Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: your memory page size is 4096 bytes Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: detected max file descriptor number: 2048 Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: lock engine: pthread robust mutexes Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: thunder lock: enabled Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 04 08:26:23.849744 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 08:26:23.889362 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: Python main interpreter initialized at 0x563619d4f9c0 Oct 04 08:26:23.889362 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: python threads support enabled Oct 04 08:26:23.889362 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: your server socket listen backlog is limited to 100 connections Oct 04 08:26:23.889362 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: your mercy for graceful operations on workers is 90 seconds Oct 04 08:26:23.889802 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 08:26:23.889802 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: *** Operational MODE: preforking *** Oct 04 08:26:23.890078 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: *** uWSGI is running in multiple interpreter mode *** Oct 04 08:26:23.890078 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: spawned uWSGI master process (pid: 105627) Oct 04 08:26:23.890294 dv-cinder-860209-1-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 04 08:26:23.890916 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: spawned uWSGI worker 1 (pid: 105628, cores: 1) Oct 04 08:26:23.891253 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: spawned uWSGI worker 2 (pid: 105629, cores: 1) Oct 04 08:26:23.891410 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 08:26:24.557188 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 04 08:26:24.572676 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 04 08:26:25.659201 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:26:25.659201 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: from cryptography.utils import int_from_bytes Oct 04 08:26:25.659816 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:26:25.659816 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: from cryptography.utils import int_from_bytes Oct 04 08:26:25.673224 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:26:25.673224 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: from cryptography.utils import int_from_bytes Oct 04 08:26:25.673710 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:26:25.673710 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: from cryptography.utils import int_from_bytes Oct 04 08:26:28.183492 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:26:28.184065 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:26:28.184277 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:26:28.184643 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:26:28.184900 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:26:28.185286 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.185614 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.185938 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.186257 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.186609 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.186925 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.187084 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:26:28.187433 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.187527 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:26:28.187616 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.187868 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.187954 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:26:28.188128 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.188224 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:26:28.188444 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.188536 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:26:28.188769 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860209-1-iscsi {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.188862 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.189093 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.189191 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.189399 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.189500 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.189710 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.189796 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.190026 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.190119 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.190331 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.190417 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.190675 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.190847 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.190991 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.191147 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.191321 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.191486 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.191594 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.191883 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.192048 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.192211 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860209-1-iscsi {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.192372 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.192513 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.192659 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.192811 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.192950 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.193121 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.193251 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.193439 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.193532 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.193744 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.193832 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.194049 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.194133 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.194356 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.194455 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.194710 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.194900 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.195116 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860209-1-iscsi {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.195218 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.195437 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.195584 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.195777 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.195961 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.196124 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.196428 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.196544 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.196935 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.197031 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.197256 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.197352 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.197580 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.197721 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.197891 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.197986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.198200 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.198298 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.198517 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.198679 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.198863 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.198958 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860209-1-iscsi {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.199213 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.199359 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.199498 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.199596 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.199800 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.199890 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.200110 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.200246 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.200421 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.200558 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.200739 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.200870 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.201046 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.201194 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.201351 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.201499 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.201604 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.201807 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.201895 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.202150 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.202252 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.202471 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.202602 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.202819 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.202917 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.203089 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.203186 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.203340 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.203441 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.203660 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.203757 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.203978 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.204067 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.204289 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.204376 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.204599 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.204692 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.204906 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.204996 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.205162 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.205247 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.205612 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.205730 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.205836 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.205928 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.206095 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.206184 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.206405 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.206502 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.206680 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.206770 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.206935 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.207058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.207254 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.207383 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.207575 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.207694 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.207884 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.208014 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.208194 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.208415 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.208502 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.208669 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.208793 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.208981 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.209099 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.209323 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.209410 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.209642 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.209748 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.209952 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.210048 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.210273 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.210359 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.210609 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.210700 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.210922 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.211021 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.211229 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.211320 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.212173 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.212262 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.212586 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.212856 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.212946 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.213119 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.213206 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.213433 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.213529 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.213756 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.213849 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.214065 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.214159 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.214385 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.214556 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.214723 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.214822 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.215036 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.215123 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.215344 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.215439 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.215646 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.215786 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.215953 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.216038 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.216259 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.216356 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.216570 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.216673 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.216879 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.216975 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.217129 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.217216 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.217432 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.217518 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.217736 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.217824 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.218055 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.218143 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.218359 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.218446 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.218688 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.218793 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.218995 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.219083 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.219297 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.219383 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.219605 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.219691 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.219909 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.219996 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.220212 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.220300 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.220516 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.220603 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.220819 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.220905 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.221124 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.221249 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.221425 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.221511 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.221727 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.221814 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.222031 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.222118 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.222332 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.222428 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.222678 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.222765 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.222989 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.223076 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.223294 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.223392 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.223600 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.223695 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.223900 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.223987 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.224203 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.224300 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.224518 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.224603 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.224823 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.224908 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.225126 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.225219 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.225430 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.225517 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.225733 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.225873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.226034 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.226136 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.226337 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.226425 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.226664 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.226767 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.227008 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.227660 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.227873 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.228091 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.228188 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.228444 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.228554 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.228754 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.228865 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.229064 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.229188 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.229372 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.229500 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.229693 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.229812 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.230032 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.230119 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.230369 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.230467 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.230734 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.230822 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.231044 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.231133 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.231349 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.231445 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.231657 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.231748 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.231962 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.232049 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.232284 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.232373 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.232608 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.232696 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.232919 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.233013 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.233229 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.233318 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.233535 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.233622 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.233845 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.233938 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:26:28.234166 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.234255 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.234486 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.234621 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.234845 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.234947 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.235164 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.235251 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.235481 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.235566 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.235803 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.235940 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.236117 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.236212 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.236437 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.236533 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.236753 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.236850 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.237074 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.237161 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.237385 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.237470 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.237697 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.237798 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.238008 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.238095 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.238368 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.238477 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.238671 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.238775 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.239006 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.239094 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.239318 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.239412 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.239628 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.239725 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.239942 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.240030 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.240250 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.240344 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.240571 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.240665 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.240884 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.240971 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.241206 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.241293 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.241517 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.241613 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.241836 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.241971 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.242151 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.242285 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.242471 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.242673 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.242866 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.243140 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.243274 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.243450 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.243593 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.243785 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.243922 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.244101 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.244240 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.244428 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.244570 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.244745 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.244884 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.245058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.245205 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.245377 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.245518 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.245691 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.245825 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.246003 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.246137 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.246316 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.246452 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.246644 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.246797 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.246946 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.247033 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.247236 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.247334 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.247549 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.247644 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.247852 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.247938 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.248173 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.248261 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.248488 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.248574 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.248802 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.248897 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.249114 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.249201 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.249428 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.249516 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.249739 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.249838 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.250048 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.250142 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.250635 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.250731 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.250947 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.251048 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.251253 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.251356 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.251562 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.251661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.251865 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.251965 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.252171 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.252257 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.252475 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.252563 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.252789 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.252876 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.253113 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.253208 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.253419 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.253513 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.253729 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.253825 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.254045 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.254132 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.254356 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.254443 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.254695 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.254796 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.255010 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.255109 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.255318 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.255406 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.255636 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.255733 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.255947 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.256033 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.256253 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.256426 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.256577 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.256654 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.256888 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.257074 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.257193 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.257291 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.257514 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.257602 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.257821 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.257918 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.258130 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.258268 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.258535 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.258650 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.259008 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.259135 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.259319 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.259524 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.259633 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.259845 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.259934 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.260159 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.260255 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.260488 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.260576 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.260805 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.260892 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.261119 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.261214 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.261422 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.261516 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.261726 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.261823 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.262053 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.262150 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.262372 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.262471 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.262703 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.262792 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.263011 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.263112 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.263318 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.263424 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.263626 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.263713 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.263934 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.264022 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.264238 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.264335 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.264543 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.264644 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.264851 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.264949 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.265157 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.265253 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.265465 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.265562 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.265774 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.265863 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.266078 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.266167 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.266383 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.266472 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.266765 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.266868 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.267038 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.267220 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.267369 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.267480 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.267647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.267736 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.267951 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.268048 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.268276 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.268373 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.268588 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.268688 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.268903 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.268989 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.269224 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.269321 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.269531 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.269628 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.269848 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.269936 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.270171 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.270266 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.270487 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.270609 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.270824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.270913 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.271135 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.271247 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.271447 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.271536 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.271760 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.271886 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.272069 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.272155 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.272379 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.272464 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.272691 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.272785 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.272998 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.273086 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.273315 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.273400 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.273625 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.273719 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.274717 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.274837 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.274910 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.275238 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.275340 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.275474 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.275571 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.275794 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.275881 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.276100 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.276187 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.276422 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.276575 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.276742 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.276830 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.277077 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.277183 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.277391 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.277495 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.277699 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.277800 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.278013 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.278101 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.278325 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.278413 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.278662 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.278760 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.278976 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.279076 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.279284 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.279372 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.279592 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.279687 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.279899 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.279994 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.280213 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.280300 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.280531 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.280628 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.280833 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.280930 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.281136 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.281221 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.281450 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.281543 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.281763 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.281859 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.282072 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.282170 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.282384 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.282480 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.282724 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.282843 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.283043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.283141 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.283353 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.283439 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.283667 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.283776 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.283978 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.284067 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.284282 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.284377 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.284585 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.284684 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.284896 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.284992 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.285203 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.285298 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.285507 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.285595 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.285812 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.285909 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.286116 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.286202 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.286451 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.286637 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.286754 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.286860 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.287061 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.287151 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.287367 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.287501 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.287677 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.287774 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.287984 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.288080 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.288296 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.288389 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.288605 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.288691 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.288915 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.289016 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.289219 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.289306 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.290113 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.290217 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.290450 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.290823 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.290939 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.291092 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.291180 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.291401 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.291487 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.291711 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.291798 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.292020 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.292116 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.292343 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.292449 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.292655 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.292743 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.292961 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.293056 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.293263 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.293373 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.293573 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.293675 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.293884 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.293979 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.294202 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.294464 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.294573 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.294836 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.294933 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.295152 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.295242 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.295465 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.295555 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.295798 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.295898 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.296093 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.296191 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.296404 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.296505 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.296716 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.296852 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.297025 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.297116 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.297348 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.297451 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.297658 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.297747 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.297965 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.298053 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.298272 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.298364 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.298605 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.298700 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.298916 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.299007 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.299222 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.299364 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.299533 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.299790 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.299887 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.300121 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.300221 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.300430 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.300515 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.300745 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.300835 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.301075 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.301174 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.301392 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.301489 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.301699 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.301790 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.302013 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.302104 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.302324 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.302415 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.302659 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.302749 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.302973 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.303081 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.303279 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.303368 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.303581 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.303680 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.303888 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.303986 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.304190 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.304277 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.304494 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.304585 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.304801 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.304902 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.305155 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.305247 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.305529 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.305618 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.305837 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.306105 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.306202 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.306419 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.306514 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.306765 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.306991 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.307142 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.307290 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.307442 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.307592 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.307766 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.307933 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.308094 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.308241 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.308410 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.308559 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.308716 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.308871 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.309011 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.309167 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.309303 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.309439 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.309590 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.309749 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.309892 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.310034 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.310176 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.310319 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.310476 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.310796 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.310980 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.311131 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.311295 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.311441 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.311702 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.311864 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.312012 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.312152 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.312308 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.312478 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.312640 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.312783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.312925 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.313064 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.313216 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.313354 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.313502 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.313645 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.313787 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.313932 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.314073 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.314227 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.314371 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.314512 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.314681 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.314831 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.314979 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.315146 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.315293 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.315438 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.315569 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.315708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.315848 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.316002 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.316138 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.316280 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.316419 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.316559 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.316701 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.316852 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.317028 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.317171 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.317322 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.317462 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.317614 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.317755 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.317900 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.318042 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.318183 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.318345 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.318502 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.318692 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.318837 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.318978 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.319127 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.319275 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.319434 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.319581 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.319722 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.319864 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.320009 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.320152 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.320316 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.320470 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.320610 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.321878 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.322032 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.322241 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.322394 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.322535 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.322697 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.322697 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.322930 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.323070 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.323222 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.323360 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.323496 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.323632 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.323770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.323923 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.324054 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.324177 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.324304 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.324440 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.324581 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.324734 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.324873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.325015 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.325155 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.325286 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.325439 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.325582 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.325725 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.325878 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.326022 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.326167 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.326324 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.326468 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.326666 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.326811 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.326954 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.327172 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.327339 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.327488 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.327630 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.327778 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.327917 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.328126 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.328294 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.328440 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.328586 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.328739 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.328886 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.329045 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.329198 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.329327 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.329461 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.329596 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.329731 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.329877 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.330038 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.330197 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.330343 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.330480 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.330733 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.330902 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.331043 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.331187 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.331355 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.331557 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.331699 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.331854 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.332006 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.332156 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.332310 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.332454 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.332613 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.332752 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.332897 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.333044 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.333254 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.333405 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.333569 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.333707 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.333845 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.334004 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.334151 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.334295 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.334449 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.334679 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.334823 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.334962 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.335108 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.335256 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.335422 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.335565 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.335712 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.335859 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.336004 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.336163 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.336307 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.337032 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.337238 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.337395 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.337544 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.337877 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.338018 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.338164 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.338304 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.338443 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.338608 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.338773 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.338773 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.339041 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.339184 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.339332 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.339478 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.339639 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.339779 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.339919 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.340060 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.340197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.340341 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.340499 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.340643 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.340786 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.340786 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.341058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.341318 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.341464 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.341594 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.341740 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.341884 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.342025 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.342189 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.342332 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.342482 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.342687 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.342840 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.342983 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.343136 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.343330 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.343465 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.343600 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.343735 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.343892 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.344029 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.344175 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.344175 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.344435 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.344578 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.344733 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.344878 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.345010 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.345010 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.345292 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.345436 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.345592 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.345729 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.345878 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.346018 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.346151 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.346287 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.346438 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.346602 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.346760 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.346908 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.347049 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.347205 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.347403 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.347549 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.347700 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.347840 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.347976 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.348132 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.348273 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.348491 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.348693 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.348840 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.348985 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.349137 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.349274 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.349414 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.349548 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.349697 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.349876 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350025 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350160 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350303 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350424 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350598 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350840 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.350979 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351163 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351259 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351343 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351474 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351593 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351721 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351821 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.351904 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.352035 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.352124 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.352304 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.352490 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.352668 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.352902 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.353002 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.353252 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.353390 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.353577 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.353666 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.353888 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.353977 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.354210 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.354298 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.354518 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.354633 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.354858 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.355024 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.355173 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.355270 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.355485 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.355590 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.355810 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.355908 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.356104 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.356206 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.356429 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.356517 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.356732 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.356819 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.357056 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.357152 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.357372 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.357506 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.357696 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.357792 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.358023 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.358120 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.358332 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.358422 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.358680 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.358773 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.358996 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.359086 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.359303 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.359399 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.359610 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.359699 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.359916 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.360006 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.360245 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.360332 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.360552 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.360649 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.360874 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.360961 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.361180 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.361277 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.361491 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.361580 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.361806 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.361969 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.362116 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.362314 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.362470 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.362683 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.362827 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.362966 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.363124 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.363267 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.363422 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.363559 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.363689 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.363832 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.363963 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.364091 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.364261 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.364562 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.364710 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.364847 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.364981 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.365115 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.365256 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.365400 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.365545 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.365681 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.365816 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.365950 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.366092 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.366236 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.366378 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.366523 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.366679 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.366764 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.367004 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.367105 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.367310 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.367399 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.367662 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.367765 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.367974 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.368282 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.368378 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.368629 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.368755 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.368887 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.369081 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.369231 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.369417 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.369538 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.369743 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.369867 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.370060 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.370179 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.370386 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.370504 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.370760 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.370867 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.371073 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.371170 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.371382 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.371472 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.371697 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.371794 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.372005 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.372173 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.372314 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.372403 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.372623 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.372730 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.372934 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.373023 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.373240 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.373330 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.373556 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.373744 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.373903 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.374069 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.374219 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.374432 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.374569 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.374791 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.374890 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.375104 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.375201 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.375678 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.375783 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.375989 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.376076 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.376294 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.376382 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.376609 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.376714 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.376922 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.377010 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.377236 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.377334 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.377545 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.377633 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.377854 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.377950 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.378173 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.378261 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.378489 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.378724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.378841 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.378941 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.379163 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.379250 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.379480 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.379569 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.379793 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.379879 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.380108 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.380280 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.380417 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.380513 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.380731 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.380820 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.381047 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.381145 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.381364 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.381452 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.381678 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.381925 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.382074 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.382254 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.382342 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.382585 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.382686 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.382902 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.383012 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.383771 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.383950 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.384204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.384309 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.384508 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.384713 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.384806 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.385035 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.385124 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.385354 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.385450 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.385670 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.385759 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.385983 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.386073 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.386316 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.386412 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.386682 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.386787 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.387004 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.387093 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.387328 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.387479 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.387649 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.387844 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.387977 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.388161 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.388293 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.388498 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.388643 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.388829 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.388923 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.389155 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.389257 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.389476 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.389564 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.389797 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.389890 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.390123 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.390221 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.390427 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.390523 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.390762 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.390852 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.391074 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.391181 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.391384 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.391471 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.391693 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.391784 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.392005 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.392127 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.392315 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.392437 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.392621 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.392712 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.392934 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.393026 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.393250 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.393340 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.393581 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.393782 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.393894 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.394138 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.394240 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.394452 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.394571 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.394808 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.394901 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.395136 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.395342 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.395459 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.395658 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.395759 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.395994 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.396096 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.396323 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.396411 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.396655 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.396759 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.396972 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.397064 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.397282 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.397372 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.397593 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.397694 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.397901 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.397988 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.398209 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.398302 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.398519 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.398644 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.398948 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.399566 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.399831 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.399998 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.400151 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.400423 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.400572 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.400747 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.400884 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.401062 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.401211 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.401388 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.401524 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.401703 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.401838 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.402058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.402243 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.402389 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.402602 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.402741 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.402973 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.403139 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.403277 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.403411 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.403578 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.403725 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.403887 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.404073 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.404256 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.404389 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.404524 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.404658 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.404814 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.404958 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.405124 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.405290 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.405426 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.405567 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.405741 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.405886 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.406030 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.406162 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.406340 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.406474 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.406684 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.406777 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.406998 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.407089 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.407304 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.407403 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.407617 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.407708 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.407924 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.408017 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.408251 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.408343 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.408560 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.408653 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.408886 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.409097 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.409189 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.409438 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.409538 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.409764 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.409853 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.410069 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.410159 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.410376 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.410468 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.410704 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.410794 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.411017 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.411111 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.411321 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.411417 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.411622 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.411711 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.411929 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.412020 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.412241 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.412331 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.412572 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.412661 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.412884 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.412980 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.413189 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.413290 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.413501 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.413590 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.413805 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.413900 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.414111 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.414200 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.414422 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.414610 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.414799 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.414975 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.415156 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.415426 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.415541 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.415750 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.415861 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.416071 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.416169 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.416428 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.416517 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.416754 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.416861 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.417121 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.417292 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.417441 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418011 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418143 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418263 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418361 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418489 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418595 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418724 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418810 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.418952 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.419040 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.419219 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.419313 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.419528 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.419620 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.419857 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.419957 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.420170 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.420261 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.420485 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.420585 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.420797 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.421108 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.421204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.421417 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.421514 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.421727 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.421817 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.422037 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.422173 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.422348 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.422437 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.422681 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.422775 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.422989 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.423092 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.423297 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.423397 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.423605 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.423701 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.423915 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.424010 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.424225 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.424326 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.424569 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.424668 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.424877 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.424969 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.425197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.425295 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.425516 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.425606 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.425831 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.425923 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.426149 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.426248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.426467 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.426578 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.426803 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.426895 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.427114 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.427207 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.427423 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.427522 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.427727 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.427814 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.428042 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.428134 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.428349 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.428440 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.428652 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.428750 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.428958 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.429265 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.429360 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.429577 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.429665 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.429883 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.429975 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.430230 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.430404 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.430540 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.430871 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.430983 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.431180 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.431270 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.431486 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.431574 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.431798 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.431896 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.432104 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.432232 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.432410 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.432499 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.432719 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.432819 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.433106 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.433208 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.433330 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.433437 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.433657 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.433751 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.433991 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.434083 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.434306 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.434395 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.434660 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.434756 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.434979 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.435074 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.435301 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.435402 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.435613 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.435702 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.435919 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.436010 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.436230 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.436335 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.436548 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.436635 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.436853 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.436951 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.437163 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.437268 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.437472 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.437570 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.437782 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.437870 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.438095 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.438192 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.438400 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.438494 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.438730 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.438833 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.439037 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.439128 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.439344 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.439433 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.439649 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.439739 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.439955 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.440046 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.440266 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.440357 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.440574 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.440670 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.440877 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.440969 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.441181 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.441271 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.441486 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.441576 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.441792 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.441882 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.442127 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.442223 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.442443 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.442582 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.442785 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.442886 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.443100 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.443317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.443415 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.443507 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.443717 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.443807 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.444030 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.444118 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.444341 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.444438 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.444659 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.444794 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.444975 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.445063 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.445287 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.445377 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.445616 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.446236 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.446474 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.446584 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.446834 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.447136 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.447226 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.447503 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.447593 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.447817 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.447922 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.448130 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.448261 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.448446 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.448535 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.448780 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.448879 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.449094 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.449180 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.449404 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.449500 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.449649 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:26:28.449740 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.450038 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.450360 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.450704 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.451023 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.451356 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.451671 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.451985 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.452299 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.452611 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.452925 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.453244 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.453558 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.453869 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.454186 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.454501 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.454852 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.455182 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.455494 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.455821 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.456149 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.456499 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.456812 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.457161 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.457490 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.457804 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.458118 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.458425 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:26:28.458692 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:26:28.604904 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO dbcounter [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Registered counter for database nova_api Oct 04 08:26:28.615954 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO dbcounter [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Registered counter for database nova_api Oct 04 08:26:28.624948 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_db.sqlalchemy.engines [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105628) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.631939 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writer thread running {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:28.637305 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_db.sqlalchemy.engines [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.643908 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writer thread running {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:28.690628 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.context [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),4bcedf19-741d-498b-8c07-5371abba3edf(cell1) {{(pid=105628) load_cells /opt/stack/nova/nova/context.py:464}} Oct 04 08:26:28.694456 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_concurrency.lockutils [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105628) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:26:28.694970 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_concurrency.lockutils [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105628) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:26:28.697332 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_concurrency.lockutils [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105628) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:26:28.698081 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_concurrency.lockutils [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105628) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:26:28.698526 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_concurrency.lockutils [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105628) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:26:28.700572 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_concurrency.lockutils [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105628) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:26:28.702608 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO dbcounter [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Registered counter for database nova_cell0 Oct 04 08:26:28.702832 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.context [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),4bcedf19-741d-498b-8c07-5371abba3edf(cell1) {{(pid=105629) load_cells /opt/stack/nova/nova/context.py:464}} Oct 04 08:26:28.706772 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO dbcounter [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Registered counter for database nova_cell1 Oct 04 08:26:28.706893 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:26:28.707258 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:26:28.708845 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:26:28.709579 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:26:28.710028 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:26:28.712120 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:26:28.714208 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO dbcounter [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Registered counter for database nova_cell0 Oct 04 08:26:28.715195 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_db.sqlalchemy.engines [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105628) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.716407 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_db.sqlalchemy.engines [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105628) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.718631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO dbcounter [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Registered counter for database nova_cell1 Oct 04 08:26:28.723254 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writer thread running {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:28.724780 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writer thread running {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:28.725889 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_db.sqlalchemy.engines [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.727841 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_db.sqlalchemy.engines [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.733307 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writer thread running {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:28.738457 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writer thread running {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:28.874710 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO dbcounter [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Registered counter for database nova_cell0 Oct 04 08:26:28.881801 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG oslo_db.sqlalchemy.engines [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105628) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.882308 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO dbcounter [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Registered counter for database nova_cell0 Oct 04 08:26:28.887282 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writer thread running {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:28.891771 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_db.sqlalchemy.engines [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:26:28.896500 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writer thread running {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:26:29.234971 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: WARNING keystonemiddleware.auth_token [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:26:29.237799 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: WARNING keystonemiddleware.auth_token [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:26:29.242271 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: WARNING keystonemiddleware.auth_token [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:26:29.245701 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: WARNING keystonemiddleware.auth_token [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:26:29.596493 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: WARNING keystonemiddleware.auth_token [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:26:29.603029 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: WARNING keystonemiddleware.auth_token [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:26:29.608665 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563619d4f9c0 pid: 105629 (default app) Oct 04 08:26:29.609069 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: WARNING keystonemiddleware.auth_token [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:26:29.610896 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:26:29.612085 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] 10.222.0.16 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001884 Oct 04 08:26:29.612501 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 788 bytes} [Tue Oct 4 08:26:29 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:26:29.615307 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: WARNING keystonemiddleware.auth_token [None req-de7e5387-3566-4fd1-8447-c9e6ff95f8dc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:26:29.620985 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563619d4f9c0 pid: 105628 (default app) Oct 04 08:27:53.539445 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:27:53.541093 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002117 Oct 04 08:27:53.541751 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:27:53 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:27:53.555209 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_api:SELECT=3 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:53.556195 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_cell0:SELECT=3 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:53.557258 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_cell1:SELECT=3 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:53.558134 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:54.468232 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a99e166f-e706-47a0-8a2a-50ad2bd25883 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:27:54.502246 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-a99e166f-e706-47a0-8a2a-50ad2bd25883 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.953150 Oct 04 08:27:54.502972 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 1/3] 10.222.0.16 () {60 vars in 1292 bytes} [Tue Oct 4 08:27:53 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 955 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:27:57.161565 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:27:57.162615 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-7e19320a-e973-40bc-843c-8de3a0f2992e None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001424 Oct 04 08:27:57.163718 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 3/4] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:27:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:27:57.624728 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-6a4d1b50-3b44-428a-83b0-c4b645cbde29 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "name": "cirros256", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "os-flavor-access:is_public": true, "swap": 0, "rxtx_factor": 1.0, "id": "c1", "ram": 256}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:27:57.676291 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-6a4d1b50-3b44-428a-83b0-c4b645cbde29 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.504589 Oct 04 08:27:57.676291 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 2/5] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:27:57 2022] POST /compute/v2.1/flavors => generated 407 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:27:57.689073 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_api:SELECT=3 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:57.690166 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_cell0:SELECT=3 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:57.690957 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_cell1:SELECT=3 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:57.691885 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:27:57.909966 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-da44a90e-51d3-4208-88f9-8e0e76977a40 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:27:57.981827 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-da44a90e-51d3-4208-88f9-8e0e76977a40 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.297924 Oct 04 08:27:57.982418 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 4/6] 10.222.0.16 () {64 vars in 1336 bytes} [Tue Oct 4 08:27:57 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:28:00.680788 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-6a4d1b50-3b44-428a-83b0-c4b645cbde29 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:00.683262 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-6a4d1b50-3b44-428a-83b0-c4b645cbde29 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.004514 Oct 04 08:28:00.684420 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 3/7] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:00 2022] GET /compute/v2.1 => generated 388 bytes in 7 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:01.449036 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-9c320abd-dc67-4fe4-9bc0-ee2bb9336ea7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "rxtx_factor": 1.0, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds512M", "os-flavor-access:is_public": true, "disk": 5, "ram": 512, "id": "d1"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:01.496805 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-9c320abd-dc67-4fe4-9bc0-ee2bb9336ea7 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.806325 Oct 04 08:28:01.497391 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 5/8] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:00 2022] POST /compute/v2.1/flavors => generated 404 bytes in 808 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:01.511246 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-afc392cd-1a89-4769-a790-c1039796e49d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:01.552610 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-afc392cd-1a89-4769-a790-c1039796e49d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.046343 Oct 04 08:28:01.554294 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 4/9] 10.222.0.16 () {64 vars in 1336 bytes} [Tue Oct 4 08:28:01 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:04.290292 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-9c320abd-dc67-4fe4-9bc0-ee2bb9336ea7 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:04.291439 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-9c320abd-dc67-4fe4-9bc0-ee2bb9336ea7 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001602 Oct 04 08:28:04.291882 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:04 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:04.759575 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-7d1987f0-1d70-4415-80ca-34dbc90a3434 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "id": "d2", "os-flavor-access:is_public": true, "ram": 1024, "name": "ds1G", "swap": 0, "disk": 10, "vcpus": 1}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:04.781554 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-7d1987f0-1d70-4415-80ca-34dbc90a3434 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.482716 Oct 04 08:28:04.782456 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 5/11] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:04 2022] POST /compute/v2.1/flavors => generated 404 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:04.796685 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-fd2a0764-484d-42e2-822e-017fa8f75855 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:04.827673 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-fd2a0764-484d-42e2-822e-017fa8f75855 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034851 Oct 04 08:28:04.828475 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 7/12] 10.222.0.16 () {64 vars in 1336 bytes} [Tue Oct 4 08:28:04 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:28:07.552540 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-7d1987f0-1d70-4415-80ca-34dbc90a3434 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:07.554645 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-7d1987f0-1d70-4415-80ca-34dbc90a3434 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002389 Oct 04 08:28:07.556209 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 6/13] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:07 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:08.011895 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-d69a1c56-de2e-4e03-83a2-b8c1149b3d99 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "ds2G", "disk": 10, "id": "d3", "swap": 0, "vcpus": 2, "os-flavor-access:is_public": true, "ram": 2048}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:08.033014 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-d69a1c56-de2e-4e03-83a2-b8c1149b3d99 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.471189 Oct 04 08:28:08.033808 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 8/14] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:07 2022] POST /compute/v2.1/flavors => generated 404 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:08.046499 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-3629676e-4363-454c-a95e-6aea275d7a8c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:08.075775 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-3629676e-4363-454c-a95e-6aea275d7a8c admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032864 Oct 04 08:28:08.076399 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 7/15] 10.222.0.16 () {64 vars in 1336 bytes} [Tue Oct 4 08:28:08 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:10.733519 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-d69a1c56-de2e-4e03-83a2-b8c1149b3d99 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:10.734572 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-d69a1c56-de2e-4e03-83a2-b8c1149b3d99 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001491 Oct 04 08:28:10.734987 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 9/16] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:28:11.188931 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-4336971e-a5d1-4221-8a82-9ee4cc56b6e8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 20, "id": "d4", "rxtx_factor": 1.0, "vcpus": 4, "name": "ds4G", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:11.211815 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-4336971e-a5d1-4221-8a82-9ee4cc56b6e8 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.470114 Oct 04 08:28:11.212330 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 8/17] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:10 2022] POST /compute/v2.1/flavors => generated 404 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:28:11.224483 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-41ec46de-c665-4fe3-90f0-48ca3f9f94c7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:11.252204 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-41ec46de-c665-4fe3-90f0-48ca3f9f94c7 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030684 Oct 04 08:28:11.253131 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 10/18] 10.222.0.16 () {64 vars in 1336 bytes} [Tue Oct 4 08:28:11 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:13.930707 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-4336971e-a5d1-4221-8a82-9ee4cc56b6e8 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:13.931733 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-4336971e-a5d1-4221-8a82-9ee4cc56b6e8 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Oct 04 08:28:13.932149 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 9/19] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:14.385465 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-e29716d3-ab7c-4acd-a0d5-568e7af3f62c admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:14.410886 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-e29716d3-ab7c-4acd-a0d5-568e7af3f62c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.472240 Oct 04 08:28:14.411461 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 11/20] 10.222.0.16 () {60 vars in 1292 bytes} [Tue Oct 4 08:28:13 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:28:17.061018 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-4336971e-a5d1-4221-8a82-9ee4cc56b6e8 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:17.062075 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-4336971e-a5d1-4221-8a82-9ee4cc56b6e8 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Oct 04 08:28:17.062515 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 10/21] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:17.484538 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-45bfcc70-0754-4a7a-b64b-fde3e6541b73 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 512, "swap": 0, "disk": 1, "name": "m1.tiny", "vcpus": 1, "rxtx_factor": 1.0, "id": "1"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:17.505424 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-45bfcc70-0754-4a7a-b64b-fde3e6541b73 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.436675 Oct 04 08:28:17.510623 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 12/22] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:17 2022] POST /compute/v2.1/flavors => generated 402 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:17.518870 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-9390a766-5bda-4f2d-9b46-6ca5c1364b5e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:17.546051 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-9390a766-5bda-4f2d-9b46-6ca5c1364b5e admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030841 Oct 04 08:28:17.546833 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 11/23] 10.222.0.16 () {64 vars in 1333 bytes} [Tue Oct 4 08:28:17 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:20.213677 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-45bfcc70-0754-4a7a-b64b-fde3e6541b73 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:20.214761 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-45bfcc70-0754-4a7a-b64b-fde3e6541b73 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Oct 04 08:28:20.215178 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:20.679013 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-7bdd5011-9265-4632-9492-8a220acbb730 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "os-flavor-access:is_public": true, "name": "m1.small", "id": "2", "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "rxtx_factor": 1.0, "ram": 2048}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:20.700634 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-7bdd5011-9265-4632-9492-8a220acbb730 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.478665 Oct 04 08:28:20.701197 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 12/25] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:20 2022] POST /compute/v2.1/flavors => generated 405 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:20.713417 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-7407596e-753a-4a31-b6b2-dd4a89638fa1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:20.736827 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-7407596e-753a-4a31-b6b2-dd4a89638fa1 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027123 Oct 04 08:28:20.737409 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 14/26] 10.222.0.16 () {64 vars in 1333 bytes} [Tue Oct 4 08:28:20 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:23.534788 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-7bdd5011-9265-4632-9492-8a220acbb730 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:23.536419 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-7bdd5011-9265-4632-9492-8a220acbb730 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002139 Oct 04 08:28:23.537271 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 13/27] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:23 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:28:24.007293 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-add74a87-2ea9-4eb4-89ae-f06f5056880b admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 4096, "name": "m1.medium", "disk": 40, "vcpus": 2, "swap": 0, "id": "3"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:24.034159 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-add74a87-2ea9-4eb4-89ae-f06f5056880b admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.490274 Oct 04 08:28:24.035041 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 15/28] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:23 2022] POST /compute/v2.1/flavors => generated 406 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:24.047984 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-99a0a3b6-f3ac-48c1-85ba-76405e150229 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:24.053279 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:28:24.078755 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-99a0a3b6-f3ac-48c1-85ba-76405e150229 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035096 Oct 04 08:28:24.079721 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 14/29] 10.222.0.16 () {64 vars in 1333 bytes} [Tue Oct 4 08:28:24 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:26.772622 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-add74a87-2ea9-4eb4-89ae-f06f5056880b admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:26.774295 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-add74a87-2ea9-4eb4-89ae-f06f5056880b admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002114 Oct 04 08:28:26.774745 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 16/30] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:26 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:27.237425 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-86b6e93f-b8cd-4fad-897c-0230ad4103aa admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.large", "swap": 0, "os-flavor-access:is_public": true, "vcpus": 4, "disk": 80, "id": "4", "ram": 8192, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:27.258515 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-86b6e93f-b8cd-4fad-897c-0230ad4103aa admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.477178 Oct 04 08:28:27.259371 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 15/31] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:26 2022] POST /compute/v2.1/flavors => generated 405 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:27.271518 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-627eb935-cf90-438c-b840-c97d4b9b00cd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:27.297263 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-627eb935-cf90-438c-b840-c97d4b9b00cd admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029761 Oct 04 08:28:27.318666 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 17/32] 10.222.0.16 () {64 vars in 1333 bytes} [Tue Oct 4 08:28:27 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:29.937853 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-86b6e93f-b8cd-4fad-897c-0230ad4103aa admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:29.938913 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-86b6e93f-b8cd-4fad-897c-0230ad4103aa admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Oct 04 08:28:29.946583 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 16/33] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:29 2022] GET /compute/v2.1 => generated 388 bytes in 9 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:30.452970 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-31a7ea2b-1943-4f32-8cbd-703dbba5d1fe admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "5", "os-flavor-access:is_public": true, "vcpus": 8, "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.xlarge", "ram": 16384, "rxtx_factor": 1.0, "swap": 0}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:30.465281 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:28:30.479960 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-31a7ea2b-1943-4f32-8cbd-703dbba5d1fe admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.532370 Oct 04 08:28:30.480681 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 18/34] 10.222.0.16 () {64 vars in 1283 bytes} [Tue Oct 4 08:28:29 2022] POST /compute/v2.1/flavors => generated 408 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:30.492951 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-297660d3-d69d-4834-b18e-be1249515e28 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:30.519184 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-297660d3-d69d-4834-b18e-be1249515e28 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029885 Oct 04 08:28:30.519793 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 17/35] 10.222.0.16 () {64 vars in 1333 bytes} [Tue Oct 4 08:28:30 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:28:47.843464 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-31a7ea2b-1943-4f32-8cbd-703dbba5d1fe admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:47.844396 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-31a7ea2b-1943-4f32-8cbd-703dbba5d1fe admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001541 Oct 04 08:28:47.844795 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 19/36] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:28:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:47.856088 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:28:48.246408 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-d1386645-c16b-46e8-ae3b-b33859badcf3 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:48.265540 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-d1386645-c16b-46e8-ae3b-b33859badcf3 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.413855 Oct 04 08:28:48.266088 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1292 bytes} [Tue Oct 4 08:28:47 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:28:49.601388 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-31a7ea2b-1943-4f32-8cbd-703dbba5d1fe admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:49.602383 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-31a7ea2b-1943-4f32-8cbd-703dbba5d1fe admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 Oct 04 08:28:49.602814 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 20/38] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:28:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:50.126677 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-c4e816c1-3979-43b2-964c-b09a37d876e9 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:50.132740 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.wsgi [None req-c4e816c1-3979-43b2-964c-b09a37d876e9 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-c4e816c1-3979-43b2-964c-b09a37d876e9 admin admin] Request method failure captured: Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Accept: application/json Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Accept-Encoding: gzip, deflate Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Connection: keep-alive Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Content-Length: 0 Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Host: 10.222.0.16 Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Openstack-Api-Version: compute 2.92 Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Openstack-System-Scope: None Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: User-Agent: python-novaclient Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Auth-Token: gAAAAABjO-7BvDoROvh0mrfDmVUEpBksODC9KURCTFmpHPJkEVPf2ieTkS0g4TZ24ITRTbvyiNbSbtDNIVIMhFj_8Wa7rpwMghO2HcByQ1pOUZCpvT18c3Wi1Jm4rw0aEseIU9MlHWCgJKIldw_kQfw-rDPeg2p698IefIpXfMjmeLtOnooKwGM Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Domain-Id: None Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Domain-Name: None Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Identity-Status: Confirmed Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Is-Admin-Project: True Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Openstack-Nova-Api-Version: 2.92 Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Domain-Id: default Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Domain-Name: Default Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Name: admin Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Role: member,admin,reader Oct 04 08:28:50.135995 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Roles: member,admin,reader Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dc7bdc31e1404eecb2eaeb287aaeab06", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant: admin Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant-Name: admin Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User: admin Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Domain-Id: default Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Domain-Name: Default Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Id: a17a7c6f52b74569b1779d66c4e1a9d1 Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Name: admin Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: method: > Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: exception: Flavor 42 could not be found. Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: action_args: {'id': '42'} Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:28:50.137724 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:50.139043 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.140720 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-c4e816c1-3979-43b2-964c-b09a37d876e9 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105628) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:28:50.142643 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-c4e816c1-3979-43b2-964c-b09a37d876e9 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.398728 Oct 04 08:28:50.142643 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 19/39] 10.222.0.16 () {60 vars in 1190 bytes} [Tue Oct 4 08:28:49 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 400 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 04 08:28:50.148075 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-e509ee81-80a4-4165-a985-6838f883c844 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:50.172033 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-e509ee81-80a4-4165-a985-6838f883c844 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.028286 Oct 04 08:28:50.172592 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 21/40] 10.222.0.16 () {60 vars in 1210 bytes} [Tue Oct 4 08:28:50 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:28:50.183370 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-51fc46bf-f147-42e0-a112-343d8bedaece admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:50.205538 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-51fc46bf-f147-42e0-a112-343d8bedaece admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026703 Oct 04 08:28:50.206085 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 20/41] 10.222.0.16 () {60 vars in 1210 bytes} [Tue Oct 4 08:28:50 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:28:50.216364 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-b79784ab-9686-4d0f-8f76-fa2553583e36 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:50.221648 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.wsgi [None req-b79784ab-9686-4d0f-8f76-fa2553583e36 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-b79784ab-9686-4d0f-8f76-fa2553583e36 admin admin] Request method failure captured: Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Accept: application/json Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Accept-Encoding: gzip, deflate Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Connection: keep-alive Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Content-Length: 0 Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Host: 10.222.0.16 Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Openstack-Api-Version: compute 2.92 Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Openstack-System-Scope: None Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: User-Agent: python-novaclient Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Auth-Token: gAAAAABjO-7BvDoROvh0mrfDmVUEpBksODC9KURCTFmpHPJkEVPf2ieTkS0g4TZ24ITRTbvyiNbSbtDNIVIMhFj_8Wa7rpwMghO2HcByQ1pOUZCpvT18c3Wi1Jm4rw0aEseIU9MlHWCgJKIldw_kQfw-rDPeg2p698IefIpXfMjmeLtOnooKwGM Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Domain-Id: None Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Domain-Name: None Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Identity-Status: Confirmed Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Is-Admin-Project: True Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Openstack-Nova-Api-Version: 2.92 Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Domain-Id: default Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Domain-Name: Default Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Name: admin Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Role: member,admin,reader Oct 04 08:28:50.225770 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Roles: member,admin,reader Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dc7bdc31e1404eecb2eaeb287aaeab06", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant: admin Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant-Name: admin Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User: admin Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Domain-Id: default Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Domain-Name: Default Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Id: a17a7c6f52b74569b1779d66c4e1a9d1 Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Name: admin Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: method: > Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: exception: Flavor 42 could not be found. Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: action_args: {'id': '42'} Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:28:50.227317 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:50.228197 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:50.229267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-b79784ab-9686-4d0f-8f76-fa2553583e36 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105629) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:28:50.230267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-b79784ab-9686-4d0f-8f76-fa2553583e36 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014692 Oct 04 08:28:50.230267 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 22/42] 10.222.0.16 () {60 vars in 1190 bytes} [Tue Oct 4 08:28:50 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 04 08:28:51.470260 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-51fc46bf-f147-42e0-a112-343d8bedaece admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:51.471460 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-51fc46bf-f147-42e0-a112-343d8bedaece admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001857 Oct 04 08:28:51.471860 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 21/43] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:28:51 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:52.023970 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-39d77225-a845-4397-80be-da651a3709bf admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:52.028372 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.wsgi [None req-39d77225-a845-4397-80be-da651a3709bf admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-39d77225-a845-4397-80be-da651a3709bf admin admin] Request method failure captured: Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Accept: application/json Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Accept-Encoding: gzip, deflate Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Connection: keep-alive Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Content-Length: 0 Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Host: 10.222.0.16 Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Openstack-Api-Version: compute 2.92 Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Openstack-System-Scope: None Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: User-Agent: python-novaclient Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Auth-Token: gAAAAABjO-7DCOfI0iXAcgJuXsVRH5zTSfY8pyPNeBnq6Y8iLWQoYWO95mwd50yq1zF-7elaGg3VI1nnJwdkQ7Wu_ghVBfsrJUAJueENMOgQcQ3v7fLvMfiDqU9gJeaa6BX3JNn9tsi3b_eya4bcfW3KB2lqOGzMqHQuEGFwQHQha9jCVWOOWsc Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Domain-Id: None Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Domain-Name: None Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Identity-Status: Confirmed Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Is-Admin-Project: True Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Openstack-Nova-Api-Version: 2.92 Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Domain-Id: default Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Domain-Name: Default Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Name: admin Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Role: member,admin,reader Oct 04 08:28:52.029873 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Roles: member,admin,reader Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dc7bdc31e1404eecb2eaeb287aaeab06", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant: admin Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant-Name: admin Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User: admin Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Domain-Id: default Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Domain-Name: Default Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Id: a17a7c6f52b74569b1779d66c4e1a9d1 Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Name: admin Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: method: > Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: exception: Flavor 84 could not be found. Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: action_args: {'id': '84'} Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:28:52.031631 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:52.032505 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.033824 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-39d77225-a845-4397-80be-da651a3709bf admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105629) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:28:52.035126 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-39d77225-a845-4397-80be-da651a3709bf admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.409382 Oct 04 08:28:52.035126 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 23/44] 10.222.0.16 () {60 vars in 1190 bytes} [Tue Oct 4 08:28:51 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 410 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 04 08:28:52.041678 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-7cdfecf7-bca4-4fc3-8735-ef1e4da144ef admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:52.056725 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-7cdfecf7-bca4-4fc3-8735-ef1e4da144ef admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018590 Oct 04 08:28:52.057273 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 22/45] 10.222.0.16 () {60 vars in 1210 bytes} [Tue Oct 4 08:28:52 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:28:52.067287 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-f55409e7-0afb-4c5c-b6d5-551136480d30 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:52.083303 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-f55409e7-0afb-4c5c-b6d5-551136480d30 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018887 Oct 04 08:28:52.083847 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1210 bytes} [Tue Oct 4 08:28:52 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:28:52.093251 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-e5560b61-0bc5-4ab5-bc5a-272329ae902d admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:52.098968 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.wsgi [None req-e5560b61-0bc5-4ab5-bc5a-272329ae902d admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-e5560b61-0bc5-4ab5-bc5a-272329ae902d admin admin] Request method failure captured: Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Accept: application/json Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Accept-Encoding: gzip, deflate Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Connection: keep-alive Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Content-Length: 0 Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Host: 10.222.0.16 Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Openstack-Api-Version: compute 2.92 Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Openstack-System-Scope: None Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: User-Agent: python-novaclient Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Auth-Token: gAAAAABjO-7DCOfI0iXAcgJuXsVRH5zTSfY8pyPNeBnq6Y8iLWQoYWO95mwd50yq1zF-7elaGg3VI1nnJwdkQ7Wu_ghVBfsrJUAJueENMOgQcQ3v7fLvMfiDqU9gJeaa6BX3JNn9tsi3b_eya4bcfW3KB2lqOGzMqHQuEGFwQHQha9jCVWOOWsc Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Domain-Id: None Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Domain-Name: None Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Identity-Status: Confirmed Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Is-Admin-Project: True Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Openstack-Nova-Api-Version: 2.92 Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Domain-Id: default Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Domain-Name: Default Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Name: admin Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Role: member,admin,reader Oct 04 08:28:52.100466 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Roles: member,admin,reader Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dc7bdc31e1404eecb2eaeb287aaeab06", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant: admin Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant-Name: admin Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User: admin Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Domain-Id: default Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Domain-Name: Default Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Id: a17a7c6f52b74569b1779d66c4e1a9d1 Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Name: admin Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: method: > Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: exception: Flavor 84 could not be found. Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: action_args: {'id': '84'} Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:28:52.102058 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:52.102986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:52.104218 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-e5560b61-0bc5-4ab5-bc5a-272329ae902d admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105628) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:28:52.105237 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-e5560b61-0bc5-4ab5-bc5a-272329ae902d admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.012164 Oct 04 08:28:52.105237 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 23/47] 10.222.0.16 () {60 vars in 1190 bytes} [Tue Oct 4 08:28:52 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 04 08:28:53.366766 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-f55409e7-0afb-4c5c-b6d5-551136480d30 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:53.367767 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-f55409e7-0afb-4c5c-b6d5-551136480d30 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001509 Oct 04 08:28:53.368171 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 25/48] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:28:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:53.898493 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a385db61-4ae8-4b2d-b760-361df97774c2 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:53.905122 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.wsgi [None req-a385db61-4ae8-4b2d-b760-361df97774c2 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a385db61-4ae8-4b2d-b760-361df97774c2 admin admin] Request method failure captured: Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Accept: application/json Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Accept-Encoding: gzip, deflate Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Connection: keep-alive Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Content-Length: 0 Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Host: 10.222.0.16 Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Openstack-Api-Version: compute 2.92 Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Openstack-System-Scope: None Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: User-Agent: python-novaclient Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Auth-Token: gAAAAABjO-7FW0yUTWWm0qRxk9gotRC0i8BjYGiQPQKCbHR-L3j5JkJWXBn-DAipRng9F3g1M1gAhQDRCnod5plCUwUpnvljw1J97pGnwIv4IEt21R4o_zJ4aiZwoy9lXCw977B2wrqgR2EE7ZY4E6xai4eUKiNscw-0L0L3vZtg2YgXqiq4Z9s Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Domain-Id: None Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Domain-Name: None Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Identity-Status: Confirmed Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Is-Admin-Project: True Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Openstack-Nova-Api-Version: 2.92 Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Domain-Id: default Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Domain-Name: Default Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Project-Name: admin Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Role: member,admin,reader Oct 04 08:28:53.906661 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Roles: member,admin,reader Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dc7bdc31e1404eecb2eaeb287aaeab06", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant: admin Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-Tenant-Name: admin Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User: admin Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Domain-Id: default Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Domain-Name: Default Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Id: a17a7c6f52b74569b1779d66c4e1a9d1 Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: X-User-Name: admin Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: method: > Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: exception: Flavor 451 could not be found. Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: action_args: {'id': '451'} Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:28:53.908204 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:53.909248 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.911388 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a385db61-4ae8-4b2d-b760-361df97774c2 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105628) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:28:53.912655 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-a385db61-4ae8-4b2d-b760-361df97774c2 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.402144 Oct 04 08:28:53.912655 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 24/49] 10.222.0.16 () {60 vars in 1193 bytes} [Tue Oct 4 08:28:53 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 403 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 04 08:28:53.919001 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-249cae4b-7f55-45f1-80df-32b7a8e643c1 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:53.934967 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-249cae4b-7f55-45f1-80df-32b7a8e643c1 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019466 Oct 04 08:28:53.935500 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 26/50] 10.222.0.16 () {60 vars in 1210 bytes} [Tue Oct 4 08:28:53 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:28:53.944996 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:53.960211 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018867 Oct 04 08:28:53.960758 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 25/51] 10.222.0.16 () {60 vars in 1210 bytes} [Tue Oct 4 08:28:53 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:28:53.970096 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-c521b312-09fa-4d1f-b586-7427d02209df admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:53.975660 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.wsgi [None req-c521b312-09fa-4d1f-b586-7427d02209df admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-c521b312-09fa-4d1f-b586-7427d02209df admin admin] Request method failure captured: Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Accept: application/json Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Accept-Encoding: gzip, deflate Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Connection: keep-alive Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Content-Length: 0 Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Host: 10.222.0.16 Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Openstack-Api-Version: compute 2.92 Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Openstack-System-Scope: None Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: User-Agent: python-novaclient Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Auth-Token: gAAAAABjO-7FW0yUTWWm0qRxk9gotRC0i8BjYGiQPQKCbHR-L3j5JkJWXBn-DAipRng9F3g1M1gAhQDRCnod5plCUwUpnvljw1J97pGnwIv4IEt21R4o_zJ4aiZwoy9lXCw977B2wrqgR2EE7ZY4E6xai4eUKiNscw-0L0L3vZtg2YgXqiq4Z9s Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Domain-Id: None Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Domain-Name: None Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Identity-Status: Confirmed Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Is-Admin-Project: True Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Openstack-Nova-Api-Version: 2.92 Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Domain-Id: default Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Domain-Name: Default Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Project-Name: admin Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Role: member,admin,reader Oct 04 08:28:53.977199 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Roles: member,admin,reader Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dc7bdc31e1404eecb2eaeb287aaeab06"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dc7bdc31e1404eecb2eaeb287aaeab06", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant: admin Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant-Id: dc7bdc31e1404eecb2eaeb287aaeab06 Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-Tenant-Name: admin Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User: admin Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Domain-Id: default Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Domain-Name: Default Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Id: a17a7c6f52b74569b1779d66c4e1a9d1 Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: X-User-Name: admin Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: method: > Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: exception: Flavor 451 could not be found. Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: action_args: {'id': '451'} Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:28:53.978783 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:53.979647 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: ERROR nova.api.openstack.wsgi Oct 04 08:28:53.980708 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-c521b312-09fa-4d1f-b586-7427d02209df admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105629) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:28:53.981750 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-c521b312-09fa-4d1f-b586-7427d02209df admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012024 Oct 04 08:28:53.981750 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 27/52] 10.222.0.16 () {60 vars in 1193 bytes} [Tue Oct 4 08:28:53 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 04 08:28:55.203325 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:55.204435 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001543 Oct 04 08:28:55.204832 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 26/53] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:28:55 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:55.732358 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-e61c2b7a-affe-475b-b0f5-50f0f9a2ec90 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:55.752330 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-e61c2b7a-affe-475b-b0f5-50f0f9a2ec90 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.408239 Oct 04 08:28:55.752884 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 28/54] 10.222.0.16 () {64 vars in 1235 bytes} [Tue Oct 4 08:28:55 2022] POST /compute/v2.1/flavors => generated 404 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:56.978817 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:56.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001476 Oct 04 08:28:56.980255 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:28:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:57.511138 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-efc334a9-6eb4-4645-aa1d-aaebf2924edb admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:57.529715 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-efc334a9-6eb4-4645-aa1d-aaebf2924edb admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.407543 Oct 04 08:28:57.530257 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 29/56] 10.222.0.16 () {64 vars in 1235 bytes} [Tue Oct 4 08:28:57 2022] POST /compute/v2.1/flavors => generated 405 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:28:58.785970 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:28:58.787340 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002553 Oct 04 08:28:58.787788 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 28/57] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:28:58 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:28:59.319095 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-3c1da7d4-d32d-4e37-985c-af7820226ed3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:28:59.337456 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-3c1da7d4-d32d-4e37-985c-af7820226ed3 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.409180 Oct 04 08:28:59.338009 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 30/58] 10.222.0.16 () {64 vars in 1235 bytes} [Tue Oct 4 08:28:58 2022] POST /compute/v2.1/flavors => generated 407 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:29:00.591365 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:29:00.592383 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Oct 04 08:29:00.592777 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 29/59] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:29:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:29:01.120213 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-dccace70-1f52-4271-a5c0-44a8ba4267fd admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:29:01.139404 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-dccace70-1f52-4271-a5c0-44a8ba4267fd admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.407713 Oct 04 08:29:01.140080 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 31/60] 10.222.0.16 () {64 vars in 1235 bytes} [Tue Oct 4 08:29:00 2022] POST /compute/v2.1/flavors => generated 409 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:29:02.384436 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:29:02.385459 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001454 Oct 04 08:29:02.385860 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 30/61] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:29:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:29:02.943261 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-80c0ddd1-f06e-4892-9287-b79aaec76e19 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:29:02.965721 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-80c0ddd1-f06e-4892-9287-b79aaec76e19 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.437088 Oct 04 08:29:02.966299 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 32/62] 10.222.0.16 () {60 vars in 1202 bytes} [Tue Oct 4 08:29:02 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:30:01.203652 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:30:01.204703 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-17f076f9-f79a-4c9e-bc61-7de4f4b4b211 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001496 Oct 04 08:30:01.205101 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 31/63] 10.222.0.16 () {54 vars in 931 bytes} [Tue Oct 4 08:30:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:30:01.216908 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:30:01.630339 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-af66e81d-74fe-4984-a9bc-21a5f492b98c admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:30:01.651237 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-af66e81d-74fe-4984-a9bc-21a5f492b98c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.438758 Oct 04 08:30:01.651876 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 33/64] 10.222.0.16 () {60 vars in 1292 bytes} [Tue Oct 4 08:30:01 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:30:01.673795 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_api:SELECT=14 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:30:01.677644 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-628e302e-f8de-4702-80e2-61920461a0cb admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:30:01.685036 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-628e302e-f8de-4702-80e2-61920461a0cb admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013303 Oct 04 08:30:01.685583 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 32/65] 10.222.0.16 () {60 vars in 1283 bytes} [Tue Oct 4 08:30:01 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:30:01.698597 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-03351492-b2b1-4719-a885-4ebfd06a4186 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:30:01.706448 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-03351492-b2b1-4719-a885-4ebfd06a4186 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011501 Oct 04 08:30:01.707179 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 34/66] 10.222.0.16 () {60 vars in 1286 bytes} [Tue Oct 4 08:30:01 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:30:01.719202 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:30:01.728857 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013279 Oct 04 08:30:01.729409 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 33/67] 10.222.0.16 () {60 vars in 1286 bytes} [Tue Oct 4 08:30:01 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:30:01.741897 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-54b323a5-198d-4ff2-9b39-a75268fecdc0 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:30:01.749225 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-54b323a5-198d-4ff2-9b39-a75268fecdc0 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010938 Oct 04 08:30:01.750018 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 35/68] 10.222.0.16 () {60 vars in 1283 bytes} [Tue Oct 4 08:30:01 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:33:06.530079 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:06.531296 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001661 Oct 04 08:33:06.531861 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 34/69] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:33:06 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:33:06.692761 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_api:SELECT=6 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.906577 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:06.912408 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:06.913126 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:06.913663 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:06.914540 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:06.915130 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:06.915655 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:06.953534 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-2fa9308b-e167-48e9-8dc6-c50d6df49019 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.261852 Oct 04 08:33:06.954283 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 36/70] 10.222.0.16 () {60 vars in 1193 bytes} [Tue Oct 4 08:33:06 2022] GET /compute/v2.1/os-services => generated 791 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:33:08.208470 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:08.209620 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Oct 04 08:33:08.210068 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 35/71] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:33:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:33:08.408233 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:08.411741 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:08.412218 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:08.412680 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:08.413391 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:08.413836 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:08.414285 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:08.434071 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-8375d72f-f220-418d-9567-e1212337c828 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.081432 Oct 04 08:33:08.434702 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 37/72] 10.222.0.16 () {60 vars in 1193 bytes} [Tue Oct 4 08:33:08 2022] GET /compute/v2.1/os-services => generated 791 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:33:20.027137 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:20.028296 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001772 Oct 04 08:33:20.028810 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 36/73] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:33:20 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:33:20.179827 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_cell0:SELECT=4 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:20.180573 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_cell1:SELECT=4 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:20.181854 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG dbcounter [-] [105629] Writing DB stats nova_api:SELECT=8 {{(pid=105629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:20.243174 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:20.247412 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:20.247890 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:20.248321 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:20.249306 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:20.249775 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:20.250174 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:20.278242 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-3b66726e-24d2-4674-b60b-e31dc230230a admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.103183 Oct 04 08:33:20.279038 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 38/74] 10.222.0.16 () {60 vars in 1193 bytes} [Tue Oct 4 08:33:20 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:33:21.472557 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG nova.api.openstack.wsgi [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] Calling method '>' {{(pid=105628) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:21.473578 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: INFO nova.api.openstack.requestlog [None req-a8ed0a52-30cb-4ecb-9143-612567aef848 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001542 Oct 04 08:33:21.473986 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: [pid: 105628|app: 0|req: 37/75] 10.222.0.16 () {56 vars in 1074 bytes} [Tue Oct 4 08:33:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:33:21.681989 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG nova.api.openstack.wsgi [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] Calling method '>' {{(pid=105629) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:33:21.686478 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:21.687009 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:21.687453 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:21.688240 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:33:21.688729 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:33:21.689156 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: DEBUG oslo_concurrency.lockutils [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105629) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:33:21.713427 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: INFO nova.api.openstack.requestlog [None req-85148a12-7f1a-42cf-bca0-b01cb63591ed admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.088919 Oct 04 08:33:21.714042 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: [pid: 105629|app: 0|req: 39/76] 10.222.0.16 () {60 vars in 1193 bytes} [Tue Oct 4 08:33:21 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:34:21.165309 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: Tue Oct 4 08:34:21 2022 - graceful shutdown triggered... Oct 04 08:34:21.164908 dv-cinder-860209-1-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 04 08:34:21.167044 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: Gracefully killing worker 1 (pid: 105628)... Oct 04 08:34:21.167712 dv-cinder-860209-1-iscsi devstack@n-api.service[105629]: Gracefully killing worker 2 (pid: 105629)... Oct 04 08:34:21.305190 dv-cinder-860209-1-iscsi devstack@n-api.service[105628]: DEBUG dbcounter [-] [105628] Writing DB stats nova_api:SELECT=4 {{(pid=105628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:22.167889 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: worker 1 buried after 1 seconds Oct 04 08:34:22.168467 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: worker 2 buried after 1 seconds Oct 04 08:34:22.168467 dv-cinder-860209-1-iscsi devstack@n-api.service[105627]: goodbye to uWSGI. Oct 04 08:34:22.169459 dv-cinder-860209-1-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 04 08:34:22.170242 dv-cinder-860209-1-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 04 08:34:22.173856 dv-cinder-860209-1-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 04 08:34:22.190129 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 08:34:22 2022] *** Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: nodename: dv-cinder-860209-1-iscsi Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: machine: x86_64 Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: clock source: unix Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: pcre jit disabled Oct 04 08:34:22.193662 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: detected number of CPU cores: 4 Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: current working directory: / Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: detected binary path: /usr/bin/uwsgi-core Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: your processes number limit is 47680 Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: your memory page size is 4096 bytes Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: detected max file descriptor number: 2048 Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: lock engine: pthread robust mutexes Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: thunder lock: enabled Oct 04 08:34:22.194253 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 04 08:34:22.194859 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 08:34:22.270075 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: Python main interpreter initialized at 0x5565527b69c0 Oct 04 08:34:22.270075 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: python threads support enabled Oct 04 08:34:22.270075 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: your server socket listen backlog is limited to 100 connections Oct 04 08:34:22.270075 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: your mercy for graceful operations on workers is 90 seconds Oct 04 08:34:22.270639 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 08:34:22.270804 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: *** Operational MODE: preforking *** Oct 04 08:34:22.271308 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: *** uWSGI is running in multiple interpreter mode *** Oct 04 08:34:22.271308 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: spawned uWSGI master process (pid: 127126) Oct 04 08:34:22.271556 dv-cinder-860209-1-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 04 08:34:22.272823 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: spawned uWSGI worker 1 (pid: 127142, cores: 1) Oct 04 08:34:22.273028 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: spawned uWSGI worker 2 (pid: 127143, cores: 1) Oct 04 08:34:22.273199 dv-cinder-860209-1-iscsi devstack@n-api.service[127126]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 08:34:23.432583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 04 08:34:23.448435 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 04 08:34:24.681353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:34:24.681353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: from cryptography.utils import int_from_bytes Oct 04 08:34:24.682013 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:34:24.682013 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: from cryptography.utils import int_from_bytes Oct 04 08:34:24.772821 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:34:24.772821 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: from cryptography.utils import int_from_bytes Oct 04 08:34:24.774256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:34:24.774256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: from cryptography.utils import int_from_bytes Oct 04 08:34:27.285875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:34:27.286785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:34:27.286785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:34:27.287083 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:34:27.287529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:34:27.287704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.287999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.289199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.289667 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.290084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.290393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.290757 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.291088 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.291427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.291691 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.292012 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.292509 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860209-1-iscsi {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.292837 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.293152 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.293474 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.293798 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.294108 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.294438 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.294783 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.295301 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.297640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.297640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.297640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.297640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.297640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.297640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.297640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.298592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.298592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.298592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860209-1-iscsi {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.299444 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.299812 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.300163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.300473 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.300888 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.300888 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.301714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.301714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.304696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.307689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.309651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.309651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.309651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.309651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.309651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.309651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.309651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.312700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.314635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.316411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.316411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.316411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.316411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.316411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.316411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.316411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.317988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.317988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.317988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.317988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.317988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.319124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.319124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.319124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.319124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.320570 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.320570 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.321089 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.321423 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.321742 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.322679 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.322679 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.322679 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.323847 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.323847 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.323847 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.324545 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.324545 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.324545 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.325206 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.325206 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.325661 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.325894 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.326122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.326401 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.326678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.326980 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.327302 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.327617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.327928 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.328251 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.328558 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.328870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.329193 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.329505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.329818 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.330129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.330426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.330765 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.331077 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.331388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.331700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.332006 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.332859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.332859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.333317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.333317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.333769 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.334010 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.334230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.334512 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.334893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.335414 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.336363 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.336725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.337054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.337373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.337700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.338033 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.338352 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.338700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.339021 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.339347 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.339673 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.340009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.340335 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.340643 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.340972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.341289 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.341617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.341951 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.342283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.344023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.344023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.344023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.344023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.344023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.345162 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.345162 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.345162 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.345162 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.346225 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.346225 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.346225 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.346947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.346947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.347442 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.347442 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.347910 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.348149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.348381 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.348630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.348903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.349302 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.349553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.349849 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.350165 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.350487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.351805 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.352624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.352953 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.353273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.354192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.354192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.354192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.355005 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.355005 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.355486 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.355486 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.356139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.356139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.356753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.356753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.357213 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.357483 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.357782 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.358023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.358265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.358628 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.358899 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.359199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.359532 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.359903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.360154 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.360457 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.360777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.361096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.361398 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.361734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.362074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.362358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.362703 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.363082 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.363380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.363692 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.364579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.364579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.365297 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.365297 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.365297 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.366070 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.366070 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.367981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.367981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.367981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.367981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.368902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.368902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.368902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.369612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.369612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.370089 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.370089 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.370586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.370845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.371156 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.371390 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.371653 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.372148 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.372403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.372649 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.372928 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.373247 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.373828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.374745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.374745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.375230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.375230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.375689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.375689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.376133 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.376366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.376782 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.377015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.377273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.377555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.377897 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.378218 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.378570 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.378888 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.379210 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.379526 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.379842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.380156 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.380472 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.380794 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.381164 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.381431 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.381728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.382995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.383262 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.383630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.383970 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.384622 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.384622 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.385366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.385366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.385830 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.386056 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.386279 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.386526 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.387258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.387582 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.387948 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.388205 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.388533 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.388851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.389173 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.389485 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.389802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.390109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.391583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.391583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.391583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.391583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.392439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.392439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.392439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.394217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.394217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.394217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.394217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.394217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.394217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.396204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.396204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.396204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.396204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.396204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.396204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.397784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.397784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.397784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.398437 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.398867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.399214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.399579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.399857 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.400181 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.400505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.400824 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.401161 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.401498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.401838 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.402079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.402403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.402753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.403104 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.403668 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.404110 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.404391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.404646 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.405569 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.405569 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.406063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.406063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.406534 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.406895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.406895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.407669 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.407988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.408313 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.408861 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.409146 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.410434 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.410434 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.410434 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.410434 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.411015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.411015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.411378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.411845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.412841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.412841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.413132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.414213 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.415187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.415522 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.415892 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.416341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.416677 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.417014 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.417699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.418061 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.418376 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.418710 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.419046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.419357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.419667 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.419976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.420295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.420608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.420915 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.421232 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.421546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.421859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.422188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.422480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.423097 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.423097 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.423564 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.423891 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.424223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.424572 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.424867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.425372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.425671 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.425983 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.426292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.426542 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.426881 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.427233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.427643 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.428210 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.428477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.428861 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.430167 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.430496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.430844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.431159 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.431482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.431793 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.432107 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.432424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.432763 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.433077 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.433389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.433707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.434020 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.434336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.434670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.434990 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.435296 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.435746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.436844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.436844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.436844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.437256 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.439041 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.439654 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.441241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.441241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.441841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.442211 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.443344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.443469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.443982 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.445181 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.445940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.446313 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.446812 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.447367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.448517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.448952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.449531 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.450440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.450841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.451663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.452296 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.452856 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.453422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.453999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.454736 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.455170 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.455723 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.456279 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.456827 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.457728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.458113 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.458893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.459451 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.460008 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.460673 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.461354 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.462052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.462535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.463126 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.463671 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.464262 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.464824 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.465588 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.466025 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.466698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.467258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.467804 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.468941 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.469499 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.470071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.470657 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.471779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.471988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.472529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.473110 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.473682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.474451 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.475503 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.475677 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.476680 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.477326 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.479009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.479592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.480175 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.480715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.481447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.482002 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.482742 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.483802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.483802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.484824 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.485032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.485505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.485975 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.486310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.486723 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.487127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.487471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.487832 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.488984 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.488984 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.489316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.489693 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.490233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.490605 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.490987 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.491354 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.491861 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.492314 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.492669 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.493180 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.493790 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.494241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.494691 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.495081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.495430 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.495783 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.496136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.496469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.496887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.497284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.497628 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.497994 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.498374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.499287 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.499598 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.500102 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.500482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.500831 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.501155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.501597 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.501990 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.502353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.502808 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.503137 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.503406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.503700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.504060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.504380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.504734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.505108 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.505428 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.505762 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.506143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.506512 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.506947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.508186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.508625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.509404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.509868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.510431 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.511332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.511850 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.512399 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.512944 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.513470 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.514015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.514537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.515128 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.515684 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.516212 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.516741 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.517291 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.517900 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.518321 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.518875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.519791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.520226 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.520785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.521205 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.521541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.522027 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.522375 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.523404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.523404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.523934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.523934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.524429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.526392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.526392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.526392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.526392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.526392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.526392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.527846 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.527846 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.528358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.528635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.528932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.529330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.529788 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.530380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.530922 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.531505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.531858 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.532383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.532970 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.533353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.533680 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.534048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.534397 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.534811 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.535257 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.535596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.536194 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.536563 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.536934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.537267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.537601 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.537928 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.538889 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.539259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.539755 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.540184 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.540625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.540911 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.541287 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.541794 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.542143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.542497 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.542889 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.543217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.543548 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.543875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.544684 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.545211 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.545563 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.545903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.546241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.546591 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.546933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.547272 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.547603 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.547886 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.548207 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.548527 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.548837 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.549165 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.549489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.549801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.550228 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.550793 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.551036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.551391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.551721 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.552057 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.552392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.552719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.553034 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.553367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.553681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.554189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.554442 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.554826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.555260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.555578 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.555875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.556201 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.556510 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.556839 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.557150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.557485 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.557825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.558121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.558436 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.558972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.559307 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.559546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.559767 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.560004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.560468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.560722 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.561174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.561559 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.561823 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.562133 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.562459 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.563229 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.564794 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.565076 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.565374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.565926 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.566256 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.566635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.567053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.567367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.567724 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.568056 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.568656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.568998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.569330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.569762 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.570282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.570697 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.571050 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.571333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.571660 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.571994 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.572282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.572600 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.572925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.573237 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.573553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.573868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.574184 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.574505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.575203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.575203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.575673 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.575909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.576144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.576377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.576742 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.577073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.577397 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.577720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.578039 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.578352 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.578715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.579035 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.579357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.579678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.580014 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.580333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.580645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.581090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.581409 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.581733 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.582060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.582379 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.582733 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.583063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.583380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.583707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.584023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.584362 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.584694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.585035 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.586047 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.586495 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.586825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.587144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.587465 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.587817 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.588106 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.588527 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.588918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.589165 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.589404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.589655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.589918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.590229 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.591338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.591338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.591338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.591845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.591845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.592144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.592446 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.592824 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.593081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.593360 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.593698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.593975 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.594303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.594681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.594981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.595303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.595870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.596451 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.596853 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.597243 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.597645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.598045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.598470 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.599073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.599487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.599876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.600265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.600658 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.601111 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.602809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.602809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.602809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.602809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.602809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.603499 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.603499 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.603756 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.604259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.604494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.604949 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.605295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.605716 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.606109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.606466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.606882 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.607274 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.607656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.608041 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.608446 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.608835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.609242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.609642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.610029 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.610430 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.610855 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.611277 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.612135 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.612135 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.612563 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.612939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.613222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.613601 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.613915 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:34:27.680698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO dbcounter [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Registered counter for database nova_api Oct 04 08:34:27.704890 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_db.sqlalchemy.engines [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:27.713293 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writer thread running {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:27.720677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:34:27.721485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:34:27.721485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:34:27.721975 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:34:27.722130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:34:27.722460 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.722815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.723140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.723463 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.723796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.724111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.724436 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.724785 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.725096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.725362 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.725684 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.726876 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-860209-1-iscsi {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.727217 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.727566 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.727876 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.728208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.728523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.728856 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.729180 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.729706 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.730021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.730332 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.730668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.731017 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.731338 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.731651 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.731977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.732289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.732624 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.732936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.733353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-860209-1-iscsi {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.733680 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.733996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.734316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.734773 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.735088 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.735399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.735730 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.736057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.736355 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.736688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.737240 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.737480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.737790 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.738029 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.738260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.738600 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.739353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.739353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.739861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.739861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.740314 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.740314 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.740778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.741019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.741252 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.741987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.743112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.743447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.743769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.744089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.744403 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.744717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.745031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.745342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.745657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.745968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.746293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.746637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.746976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.747279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.747769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.748084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.748323 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.748840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.749144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.749903 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.749903 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.750338 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.750732 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.750732 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.751188 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.751450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.751836 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.752125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.752356 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.753083 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.753083 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.753083 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.753796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.753796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.754278 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.754278 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.754860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.754860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.755307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.755531 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.755770 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.756017 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.756331 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.756646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.756956 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.758040 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.758915 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.759249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.760073 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.760073 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.760556 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.760556 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.761015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.762588 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.762588 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.762588 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.762588 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.763144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.763144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.763364 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.763583 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.763892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.764218 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.764523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.764834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.765156 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.765468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.765780 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.766086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.766397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.766731 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.767049 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.767358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.767666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.768071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.768293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.768620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.768932 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.769242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.769549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.769947 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:34:27.770213 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.770567 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.770931 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.771248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.771887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.771887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.772224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.772538 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.773675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.774441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.774817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.775140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.775458 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.775785 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.776067 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.context [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),4bcedf19-741d-498b-8c07-5371abba3edf(cell1) {{(pid=127142) load_cells /opt/stack/nova/nova/context.py:464}} Oct 04 08:34:27.776245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.776441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.776767 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.777090 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.777544 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.777717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.778050 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.778385 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.778757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.779075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.779426 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.779779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.780098 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.780425 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.780577 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:34:27.780739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.780995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:34:27.781163 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.781358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.781673 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.782021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.782336 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.782605 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:34:27.782781 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.783010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.783329 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.783488 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:34:27.783647 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.783804 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:34:27.783953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.784257 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.784572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.784888 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.785258 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.785540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.785775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:34:27.785942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.786177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.786498 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.786834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.787158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.787599 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.787873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.788034 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO dbcounter [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Registered counter for database nova_cell0 Oct 04 08:34:27.788253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.788708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.789507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.789828 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.790236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.790575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.790932 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.791273 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.791585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.791926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.792239 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.792556 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.792766 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO dbcounter [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Registered counter for database nova_cell1 Oct 04 08:34:27.792931 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.793171 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.793480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.793797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.794111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.794420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.795143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.795143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.795572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.795781 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.796182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.796604 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.799759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.805152 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.805152 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.805152 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806086 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_db.sqlalchemy.engines [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.806412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.808051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.808051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.808051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.808051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.808051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_db.sqlalchemy.engines [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.809402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.811638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.812840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.813737 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.813737 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.813737 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.814355 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writer thread running {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:27.814685 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.814685 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.814685 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.815182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.815182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.815776 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.816198 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writer thread running {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:27.816361 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.816361 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.816735 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.816893 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.817258 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.817423 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.817804 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.818062 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.818622 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.818798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.819151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.819459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.821280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.821716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.821949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.822371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.822729 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.823066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.823388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.823778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.824065 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.824337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.824652 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.824970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.825297 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.825613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.825926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.826229 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.826572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.827337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.827337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.827691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.827853 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.828109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.828432 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.828746 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.829104 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.829462 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.829732 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.829995 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.830311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.830700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.831133 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.831305 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.831717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.832039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.832624 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.832797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.832995 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.833311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.833633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.833948 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.834261 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.834608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.834935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.836034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.836927 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.837254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.837578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.837900 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.838236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.838656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.838892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.839212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.839521 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.839846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.840157 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.840858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.841264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.841428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.841719 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.842071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.842395 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.842759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.843094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.843412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.843722 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.844037 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.844347 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.844658 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.844975 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.845287 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.845598 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.845909 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.846208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.846520 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.846850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.847158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.847468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.847792 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.848101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.848403 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.848843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.849029 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.849362 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.849814 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.850118 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.850293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.850672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.851983 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.852911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.853085 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.853293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.853607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.853921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.854251 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.854611 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.854896 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.855468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.855654 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.855817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.856179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.856486 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.856820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.857026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.857318 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.857623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.857934 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.858242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.858584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.859093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.859358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.859627 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.860231 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.860231 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.860897 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.860897 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.861386 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.861386 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.862057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.862057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.863672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.863672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.863672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.863672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.863672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.863672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.865097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.865097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.865097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.865097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.866099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.866099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.866099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.866099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.867690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.868198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.868685 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.869169 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.869646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.870125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.870634 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.871106 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.871597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.872076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.872556 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.873049 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.873812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.874119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.874447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.874803 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.875173 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.875491 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.876179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.876179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.876609 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.876799 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.877127 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.877450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.877773 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.878087 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.878409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.878756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.879079 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.879387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.879705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.880033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.880352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.880672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.881005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.881332 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.881662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.881971 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.882343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.883374 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.883691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.884015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.884332 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.884654 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.884982 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.885297 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.885617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.885936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.886250 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.886582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.886893 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.887187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.887526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.887838 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.888141 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.888457 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.888767 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.889085 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.889416 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.890010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.890336 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.890810 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.890810 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.891343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.891778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.891778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.892279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.892279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.892911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.893174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.893441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.893689 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.893953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.894202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.894506 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.894845 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.895166 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.895475 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.895784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.896113 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.896423 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.896749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.897179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.897493 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.898389 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.899465 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.901170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.901170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.901170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.901170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.901170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.902794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.902794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.902794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.902794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.902794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.904123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.904123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.904123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.904123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.904123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.905365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.905365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.905365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.906115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.906115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.906115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.906873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.906873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.907352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.907352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.907855 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.908123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.908376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.908619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.908866 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.909157 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.909469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.909773 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.910084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.910676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.910971 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.911656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.911656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.911656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.912139 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.912491 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.912670 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.912882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.913211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.914819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.915015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.915374 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.915691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.916024 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.916365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.916701 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.917022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.917365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.917698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.918025 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.918342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.918687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.919019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.919331 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.919657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.919943 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.920254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.920563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.920873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.921186 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.921490 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.922086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.922086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.922857 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.922857 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.923528 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.923528 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.924026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.924026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.924516 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.924768 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.925030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.925299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.925538 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.925782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.926100 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.926418 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.926763 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.927085 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.927405 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.927721 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.928037 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.928351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.928673 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.929313 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.930075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.930672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.931084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.931370 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.931637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.932226 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.932226 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.932715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.933172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.933172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.933791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.933791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.934276 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.934533 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.934815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.935098 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.935426 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.935746 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.936067 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.936387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.936760 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.937077 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.937444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.937768 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.938091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.938413 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.939242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.939242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.939653 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.939653 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.940048 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.940385 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.940600 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.941107 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.941361 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.941612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.941872 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.942244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.942869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.943060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.943488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.943803 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.944143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.944472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.945404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.946387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.946756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.947093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.947412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.947728 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.948069 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.948467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.948778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.949190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.949468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.949802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.950133 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.950411 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.951015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.951477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.951735 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.952012 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.952345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.952769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.953160 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.953381 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.953783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.953997 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.954206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.954617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.954835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.955124 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.955440 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.955754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.956063 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.956376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.956685 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.957017 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.957329 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.957636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.957945 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.958260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.958660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.958894 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.959197 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.959549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.959810 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.960131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.961362 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.961690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.962018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.962391 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.962788 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.963051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.963457 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.963679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.964100 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.964407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.964622 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.964930 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.965249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.965564 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.965923 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.966247 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.966860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.967198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.967523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.967867 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.968204 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.968513 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.968826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.969133 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.969598 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.969813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.970074 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.970386 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.970740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.971055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.971404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.971678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.971993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.972304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.972616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.973019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.973241 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.973615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.973833 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.974230 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.974494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.974850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.975171 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.975491 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.977217 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.977217 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.977686 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.977913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.978122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.978518 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.978819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.979145 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.979455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.980835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.980835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.980835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.980835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.981832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.981832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.981832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.982730 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.982730 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.982730 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.984917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.984917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.984917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.984917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.986709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.986709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.986709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.986709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.987835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.987835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.988226 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.988723 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.989238 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.989785 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.990023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO dbcounter [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Registered counter for database nova_cell0 Oct 04 08:34:27.990246 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.990514 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.990974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.991219 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.993131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.993527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.993864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.994478 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.994478 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.994984 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.995143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.995468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.995790 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.996106 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.996422 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.996735 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.997084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.997424 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.997747 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.998076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.998703 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.999013 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.999329 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.999660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:27.999841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_db.sqlalchemy.engines [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:28.000008 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.000462 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.000699 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.000962 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.001250 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.001567 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.001882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.002225 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.002508 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.002909 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.003242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.003575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.003884 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.004404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.004575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.004847 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.005199 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.005472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.005806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.006120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.006437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.006786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.007225 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.007775 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.008459 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writer thread running {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:28.008623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.008623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.008909 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.009145 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.009469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.009900 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.010124 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.010439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.010807 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.011144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.011502 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.011819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.012177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.012509 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.012832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.013151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.013473 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:34:28.013731 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:34:28.079412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO dbcounter [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Registered counter for database nova_api Oct 04 08:34:28.103044 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_db.sqlalchemy.engines [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:28.110526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writer thread running {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:28.173446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.context [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),4bcedf19-741d-498b-8c07-5371abba3edf(cell1) {{(pid=127143) load_cells /opt/stack/nova/nova/context.py:464}} Oct 04 08:34:28.177813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:34:28.178302 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:34:28.181053 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:34:28.181953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:34:28.182338 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:34:28.184564 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:34:28.186838 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO dbcounter [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Registered counter for database nova_cell0 Oct 04 08:34:28.193974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO dbcounter [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Registered counter for database nova_cell1 Oct 04 08:34:28.203022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_db.sqlalchemy.engines [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:28.205207 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_db.sqlalchemy.engines [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:28.217860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writer thread running {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:28.222253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writer thread running {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:28.392268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO dbcounter [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Registered counter for database nova_cell0 Oct 04 08:34:28.401369 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_db.sqlalchemy.engines [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:34:28.405599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: WARNING keystonemiddleware.auth_token [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:34:28.409967 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writer thread running {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:34:28.419010 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: WARNING keystonemiddleware.auth_token [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:34:28.934193 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: WARNING keystonemiddleware.auth_token [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:34:28.946718 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: WARNING keystonemiddleware.auth_token [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:34:28.951181 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: WARNING keystonemiddleware.auth_token [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:34:28.957784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: WARNING keystonemiddleware.auth_token [None req-73964639-7430-460f-af91-9d4a54c976b9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:34:28.964228 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5565527b69c0 pid: 127142 (default app) Oct 04 08:34:29.456094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: WARNING keystonemiddleware.auth_token [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 04 08:34:29.463832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: WARNING keystonemiddleware.auth_token [None req-3de8a6bd-b726-41f1-9c3f-b311c0ec0139 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 04 08:34:29.471602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5565527b69c0 pid: 127143 (default app) Oct 04 08:35:06.947123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=3 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:06.948468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=3 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:06.949942 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=3 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:06.951882 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=3 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:08.092962 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=3 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:08.093781 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=3 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:08.095417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=3 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:08.096161 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=3 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:09.000079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8025f6a3-5f8b-4fe8-8230-2fd61c6b95bb tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Action: 'create', calling method: .version_select at 0x7f30a192f4c0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1639645181"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:10.330660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a918ee9d-ab33-4d43-a991-9a4741edbb79 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Action: 'create', calling method: .version_select at 0x7f30a192d4c0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1086444231", "metadata": {"Type": "work"}}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:12.080593 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8025f6a3-5f8b-4fe8-8230-2fd61c6b95bb tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 5.159811 Oct 04 08:35:12.082070 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 1/1] 10.222.0.16 () {60 vars in 1155 bytes} [Tue Oct 4 08:35:06 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 5163 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Oct 04 08:35:12.101837 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0c975678-e957-45e6-98ce-d97e324ed499 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a18023a0>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:12.531182 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0c975678-e957-45e6-98ce-d97e324ed499 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef" status: 200 len: 345 microversion: 2.1 time: 0.433972 Oct 04 08:35:12.532509 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 2/2] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:12 2022] GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef => generated 345 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:35:13.213738 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a918ee9d-ab33-4d43-a991-9a4741edbb79 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 5.126897 Oct 04 08:35:13.214661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 1/3] 10.222.0.16 () {60 vars in 1156 bytes} [Tue Oct 4 08:35:08 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 5129 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Oct 04 08:35:13.229893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5ac71eb7-a202-40ce-9607-c4a311262dc0 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Calling method '.version_select at 0x7f30a1802a60>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:13.547830 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-faac4b55-cfb8-49f5-8f80-16ce05092e39 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a17ff3a0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:13.606007 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5ac71eb7-a202-40ce-9607-c4a311262dc0 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac" status: 200 len: 354 microversion: 2.1 time: 0.379855 Oct 04 08:35:13.606868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 3/4] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:13 2022] GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac => generated 354 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:35:13.623418 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-242ef435-37f9-44a2-a491-3808ff4cb841 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Calling method '.version_select at 0x7f30a1798280>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:13.761181 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-faac4b55-cfb8-49f5-8f80-16ce05092e39 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef" status: 200 len: 346 microversion: 2.1 time: 0.217365 Oct 04 08:35:13.763233 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 2/5] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:13 2022] GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef => generated 346 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:13.789270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-087de200-fe1a-4483-bae0-d1d0a2743f93 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Action: 'create', calling method: .version_select at 0x7f30a17ffa60>, body: {"snapshot": {"volume_id": "863d608f-4901-42f2-a4cf-27080af3d1ef", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-422857852"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:13.860794 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-242ef435-37f9-44a2-a491-3808ff4cb841 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac" status: 200 len: 354 microversion: 2.1 time: 0.241864 Oct 04 08:35:13.879435 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 4/6] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:13 2022] GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac => generated 354 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:13.886329 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c4c29c76-1cb5-409b-a755-b9d6090e9743 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Calling method '.version_select at 0x7f30a1802ca0>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:14.066885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c4c29c76-1cb5-409b-a755-b9d6090e9743 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac" status: 202 len: 0 microversion: 2.1 time: 0.184009 Oct 04 08:35:14.067561 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 5/7] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:35:13 2022] DELETE /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac => generated 0 bytes in 188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:35:14.077019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-dd9d3815-f745-4946-ba1c-581eb71ad6d1 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Calling method '.version_select at 0x7f30a18025e0>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:14.208505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-dd9d3815-f745-4946-ba1c-581eb71ad6d1 tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac" status: 200 len: 353 microversion: 2.1 time: 0.136627 Oct 04 08:35:14.209738 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 6/8] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:14 2022] GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac => generated 353 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:35:14.224148 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-087de200-fe1a-4483-bae0-d1d0a2743f93 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.449451 Oct 04 08:35:14.225839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 3/9] 10.222.0.16 () {60 vars in 1162 bytes} [Tue Oct 4 08:35:13 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:14.236681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e0cf3579-c9e1-491c-9fae-09c46cf4fb6e tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a1798430>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:14.364231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e0cf3579-c9e1-491c-9fae-09c46cf4fb6e tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 286 microversion: 2.1 time: 0.131288 Oct 04 08:35:14.365107 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 7/10] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:14 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:14.376630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-927219ee-b1f9-47b6-af38-b848738afad5 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a1798c10>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:14.466828 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-927219ee-b1f9-47b6-af38-b848738afad5 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.094344 Oct 04 08:35:14.467613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 4/11] 10.222.0.16 () {58 vars in 1140 bytes} [Tue Oct 4 08:35:14 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:14.479885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2d831ba5-153f-4d20-a44b-c0898350cdb2 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a1802550>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:14.554258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2d831ba5-153f-4d20-a44b-c0898350cdb2 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 286 microversion: 2.1 time: 0.078012 Oct 04 08:35:14.555475 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 8/12] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:14 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 286 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:15.223071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dcbfd9dc-0336-4845-a79c-23979e87af3e tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Calling method '.version_select at 0x7f30a192dee0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:15.276145 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-dcbfd9dc-0336-4845-a79c-23979e87af3e tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] HTTP exception thrown: Volume 4244ec0e-be0a-4d14-943c-022efd3c34ac could not be found. Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dcbfd9dc-0336-4845-a79c-23979e87af3e tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Request method failure captured: Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac HTTP/1.1 Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_A73Gdm5OcTyRmjThT-_pdP_gyPrRUgHILTl8EFCN_14ovAyuMoHNSjb9hLrBRuYEelqsJsiNJa7788M0mcoaZS9smAtd193kEemUPnUJI3vFbWP6Y9Adbj6QlRnhzqFYzUp83opGDHfxKZAhJ3HNAL-XC68uD7tw3bvbjN2cyAxvY Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 7f158aa9def14740b321dacf2fbacab1 Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesGetTestJSON-2038615360 Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:35:15.281504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7f158aa9def14740b321dacf2fbacab1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7f158aa9def14740b321dacf2fbacab1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7f158aa9def14740b321dacf2fbacab1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7f158aa9def14740b321dacf2fbacab1", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesGetTestJSON-2038615360 Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 7f158aa9def14740b321dacf2fbacab1 Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesGetTestJSON-2038615360 Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesGetTestJSON-2038615360-project Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 2cf05c1f95264293ae851bb86ed6aeea Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesGetTestJSON-2038615360-project Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Volume 4244ec0e-be0a-4d14-943c-022efd3c34ac could not be found. Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '4244ec0e-be0a-4d14-943c-022efd3c34ac'} Oct 04 08:35:15.282832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4244ec0e-be0a-4d14-943c-022efd3c34ac could not be found. (HTTP 404) (Request-ID: req-70a7318b-a224-4034-9f5b-0f88ac7404cf) Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:15.283532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:15.284382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4244ec0e-be0a-4d14-943c-022efd3c34ac could not be found. Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 04 08:35:15.285271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4244ec0e-be0a-4d14-943c-022efd3c34ac could not be found. Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4244ec0e-be0a-4d14-943c-022efd3c34ac could not be found. Oct 04 08:35:15.286140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:15.289051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dcbfd9dc-0336-4845-a79c-23979e87af3e tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] Returning 404 to user: Volume 4244ec0e-be0a-4d14-943c-022efd3c34ac could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:35:15.290479 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-dcbfd9dc-0336-4845-a79c-23979e87af3e tempest-VolumesGetTestJSON-2038615360 tempest-VolumesGetTestJSON-2038615360-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac" status: 404 len: 109 microversion: 2.1 time: 0.071073 Oct 04 08:35:15.294916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 5/13] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:15 2022] GET /compute/v2.1/os-volumes/4244ec0e-be0a-4d14-943c-022efd3c34ac => generated 109 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:35:15.567999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bd64207f-aedf-40d5-acec-f18392a7f3cd tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a1802c10>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:15.640423 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bd64207f-aedf-40d5-acec-f18392a7f3cd tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 286 microversion: 2.1 time: 0.076648 Oct 04 08:35:15.641354 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 9/14] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:15 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 286 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:16.654601 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-408b3c6c-7579-42a3-b3f3-663b69219acc tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a17980d0>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:16.704785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-408b3c6c-7579-42a3-b3f3-663b69219acc tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 286 microversion: 2.1 time: 0.054636 Oct 04 08:35:16.709720 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6c5aa491-2135-4494-bda1-8c34f6461a9f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Action: 'create', calling method: .version_select at 0x7f30a215c9d0>, body: {"keypair": {"name": "tempest-keypair-499785230", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEp+tXaEK+5Cyzfqfr8hRyOkJP50ath8RoayWGg91zGHQOHpBEd1DJ95SsTuA6kZ+3Zu5fBRM87vJoyvXgsGPDzPk3c5Lf+YFNzu1CSmHlXF6riAFw+73dlIADvq9WCNIA=="}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:16.711542 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 10/15] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:16 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 286 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:16.787618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-6c5aa491-2135-4494-bda1-8c34f6461a9f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for project 3e938efbd6804a86946d80ae92e88b16. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:35:16.801726 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-6c5aa491-2135-4494-bda1-8c34f6461a9f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for user 87ca9fc2e013477cad3a3de8e8811274 and project 3e938efbd6804a86946d80ae92e88b16. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:35:16.869121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6c5aa491-2135-4494-bda1-8c34f6461a9f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.425870 Oct 04 08:35:16.883015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 6/16] 10.222.0.16 () {62 vars in 1200 bytes} [Tue Oct 4 08:35:16 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 438 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 04 08:35:17.727128 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-65727700-69eb-4a53-a524-d291ca732438 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a1802b80>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:17.828559 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-65727700-69eb-4a53-a524-d291ca732438 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 287 microversion: 2.1 time: 0.112556 Oct 04 08:35:17.838205 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 11/17] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:17 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 287 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:17.848078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-60aae53b-800c-4355-a03e-ab9cd918054c tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a16d9b80>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:18.025332 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-60aae53b-800c-4355-a03e-ab9cd918054c tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "DELETE /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 202 len: 0 microversion: 2.1 time: 0.186018 Oct 04 08:35:18.026224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 7/18] 10.222.0.16 () {58 vars in 1254 bytes} [Tue Oct 4 08:35:17 2022] DELETE /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:35:18.045669 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c51ae34a-7b98-4114-9439-df131a5ad1af tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a18024c0>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:18.123898 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c51ae34a-7b98-4114-9439-df131a5ad1af tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 286 microversion: 2.1 time: 0.091408 Oct 04 08:35:18.124540 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 12/19] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:18 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 286 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:19.137185 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0dabc2c0-2b02-4380-8106-4a88355fad3c tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a170a280>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:19.220341 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0dabc2c0-2b02-4380-8106-4a88355fad3c tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 286 microversion: 2.1 time: 0.086945 Oct 04 08:35:19.226154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 8/20] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:19 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 286 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:20.235077 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1cdb53ce-0eee-47c9-b832-b1b8c5dc8645 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a16d9d30>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:20.350659 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1cdb53ce-0eee-47c9-b832-b1b8c5dc8645 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 200 len: 286 microversion: 2.1 time: 0.119949 Oct 04 08:35:20.351632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 9/21] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:20 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 286 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:20.402801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-220447715", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "networks": [{"uuid": "5222819e-bf96-4595-9724-ed979b9e88ed"}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:20.623698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] validate_networks() for [('5222819e-bf96-4595-9724-ed979b9e88ed', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:35:21.368505 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1127efc7-8061-4c81-b7f5-229a23f56096 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a170a310>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:21.490864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-1127efc7-8061-4c81-b7f5-229a23f56096 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] HTTP exception thrown: Snapshot ea6d27dc-2b12-49f6-a6d3-3de5f964c510 could not be found. Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1127efc7-8061-4c81-b7f5-229a23f56096 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Request method failure captured: Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 HTTP/1.1 Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_A6-hBFnqmSF-PlFJpAoMW77wei_KuNeG0-oqL4Oo4aoKabzLF-rXLK3qm-IAV8pptkScTzORVrd7vbE77Tv7hYVV5foykeDDZjOKY6A3WbWUE3EAY9FJmpvIHE2J6OWeD1fS30cxPMwQyQmY4vteMkJ5teO---cbKa68_SAvW9bjE Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:35:21.493466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d61d65375ccf4967b7545338ce72bf68", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: baf5ab01e28944258aa2fdfe7951b504 Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Snapshot ea6d27dc-2b12-49f6-a6d3-3de5f964c510 could not be found. Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'ea6d27dc-2b12-49f6-a6d3-3de5f964c510'} Oct 04 08:35:21.494540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot ea6d27dc-2b12-49f6-a6d3-3de5f964c510 could not be found. (HTTP 404) (Request-ID: req-9d79b314-7e9a-41da-bacf-3628ca3411b6) Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.495200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:21.496120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot ea6d27dc-2b12-49f6-a6d3-3de5f964c510 could not be found. Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 04 08:35:21.497004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot ea6d27dc-2b12-49f6-a6d3-3de5f964c510 could not be found. Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot ea6d27dc-2b12-49f6-a6d3-3de5f964c510 could not be found. Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1127efc7-8061-4c81-b7f5-229a23f56096 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Returning 404 to user: Snapshot ea6d27dc-2b12-49f6-a6d3-3de5f964c510 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1127efc7-8061-4c81-b7f5-229a23f56096 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510" status: 404 len: 111 microversion: 2.1 time: 0.131043 Oct 04 08:35:21.497913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 10/22] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:35:21 2022] GET /compute/v2.1/os-snapshots/ea6d27dc-2b12-49f6-a6d3-3de5f964c510 => generated 111 bytes in 133 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:35:21.507417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cbb6859c-32d3-4619-b1c7-d09e1c33fe63 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a16d93a0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:21.722099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cbb6859c-32d3-4619-b1c7-d09e1c33fe63 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef" status: 202 len: 0 microversion: 2.1 time: 0.220819 Oct 04 08:35:21.723328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:35:21 2022] DELETE /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:35:21.743498 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d6e5bf4e-4a62-4187-bc32-f3b9d27e24cf tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a170ab80>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:21.938696 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-d6e5bf4e-4a62-4187-bc32-f3b9d27e24cf tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] HTTP exception thrown: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d6e5bf4e-4a62-4187-bc32-f3b9d27e24cf tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Request method failure captured: Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef HTTP/1.1 Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_A6-hBFnqmSF-PlFJpAoMW77wei_KuNeG0-oqL4Oo4aoKabzLF-rXLK3qm-IAV8pptkScTzORVrd7vbE77Tv7hYVV5foykeDDZjOKY6A3WbWUE3EAY9FJmpvIHE2J6OWeD1fS30cxPMwQyQmY4vteMkJ5teO---cbKa68_SAvW9bjE Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:35:21.941882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d61d65375ccf4967b7545338ce72bf68", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: baf5ab01e28944258aa2fdfe7951b504 Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '863d608f-4901-42f2-a4cf-27080af3d1ef'} Oct 04 08:35:21.943816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. (HTTP 404) (Request-ID: req-4877837d-f46b-42f8-a93d-53c37078353d) Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:21.944428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:21.945301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 04 08:35:21.946184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d6e5bf4e-4a62-4187-bc32-f3b9d27e24cf tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Returning 404 to user: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d6e5bf4e-4a62-4187-bc32-f3b9d27e24cf tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef" status: 404 len: 109 microversion: 2.1 time: 0.217121 Oct 04 08:35:21.962081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 12/24] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:21 2022] GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef => generated 109 bytes in 219 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:35:21.971074 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3489b19c-09fc-4229-86b3-54b4b88598ce tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a1699160>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:22.100680 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-3489b19c-09fc-4229-86b3-54b4b88598ce tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] HTTP exception thrown: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3489b19c-09fc-4229-86b3-54b4b88598ce tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Request method failure captured: Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef HTTP/1.1 Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_A6-hBFnqmSF-PlFJpAoMW77wei_KuNeG0-oqL4Oo4aoKabzLF-rXLK3qm-IAV8pptkScTzORVrd7vbE77Tv7hYVV5foykeDDZjOKY6A3WbWUE3EAY9FJmpvIHE2J6OWeD1fS30cxPMwQyQmY4vteMkJ5teO---cbKa68_SAvW9bjE Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:35:22.103742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d61d65375ccf4967b7545338ce72bf68", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: baf5ab01e28944258aa2fdfe7951b504 Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '863d608f-4901-42f2-a4cf-27080af3d1ef'} Oct 04 08:35:22.109711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. (HTTP 404) (Request-ID: req-1d929a3b-66e3-4bd6-9054-90c73846d062) Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:22.116322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 04 08:35:22.120474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 04 08:35:22.127101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.129952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.133618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3489b19c-09fc-4229-86b3-54b4b88598ce tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Returning 404 to user: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:35:22.133618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3489b19c-09fc-4229-86b3-54b4b88598ce tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef" status: 404 len: 109 microversion: 2.1 time: 0.167887 Oct 04 08:35:22.144240 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:35:21 2022] DELETE /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef => generated 109 bytes in 180 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:35:22.164658 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d81cc54a-6b76-4e17-b6a4-6c7d50605005 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Calling method '.version_select at 0x7f30a17a1af0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:22.324245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-d81cc54a-6b76-4e17-b6a4-6c7d50605005 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] HTTP exception thrown: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d81cc54a-6b76-4e17-b6a4-6c7d50605005 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Request method failure captured: Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef HTTP/1.1 Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_A6-hBFnqmSF-PlFJpAoMW77wei_KuNeG0-oqL4Oo4aoKabzLF-rXLK3qm-IAV8pptkScTzORVrd7vbE77Tv7hYVV5foykeDDZjOKY6A3WbWUE3EAY9FJmpvIHE2J6OWeD1fS30cxPMwQyQmY4vteMkJ5teO---cbKa68_SAvW9bjE Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:35:22.326748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d61d65375ccf4967b7545338ce72bf68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d61d65375ccf4967b7545338ce72bf68", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: d61d65375ccf4967b7545338ce72bf68 Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-222533834 Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: baf5ab01e28944258aa2fdfe7951b504 Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotsTestJSON-222533834-project Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '863d608f-4901-42f2-a4cf-27080af3d1ef'} Oct 04 08:35:22.328003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. (HTTP 404) (Request-ID: req-2230fb14-f235-42e0-a828-2f3f3a7f2b57) Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:22.328687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:35:22.329594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 04 08:35:22.330745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d81cc54a-6b76-4e17-b6a4-6c7d50605005 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] Returning 404 to user: Volume 863d608f-4901-42f2-a4cf-27080af3d1ef could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d81cc54a-6b76-4e17-b6a4-6c7d50605005 tempest-VolumesSnapshotsTestJSON-222533834 tempest-VolumesSnapshotsTestJSON-222533834-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef" status: 404 len: 109 microversion: 2.1 time: 0.179868 Oct 04 08:35:22.331688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 14/26] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:35:22 2022] GET /compute/v2.1/os-volumes/863d608f-4901-42f2-a4cf-27080af3d1ef => generated 109 bytes in 181 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:35:24.163694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:35:24.164810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:35:24.165783 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:35:24.166417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:35:24.167087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:35:24.226209 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:24.228599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:24.229437 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:24.230398 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:24.232844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:24.244165 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:24.374161 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-876829896", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "730a9b11-4347-4415-adec-96bd4e59e83e"}], "security_groups": [{"name": "tempest-securitygroup--1754158494"}], "key_name": "tempest-keypair-499785230", "user_data": "}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:24.383580 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Getting quotas for project 3f34eba992024f6881d1292c9f7cfd48. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:35:24.418207 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Getting quotas for user d002a844a71649d884b4ca870eb9d387 and project 3f34eba992024f6881d1292c9f7cfd48. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:35:24.461759 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:35:24.476280 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:35:24.713329 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e2671ab2-7317-4fce-b949-40fb7042fe0f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 5.146928 Oct 04 08:35:24.721730 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 13/27] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:35:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 5161 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:35:24.750808 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2f91d5c3-89a3-4e2e-b1c5-5e41a8761843 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:24.760529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2f91d5c3-89a3-4e2e-b1c5-5e41a8761843 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:24.878762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] validate_networks() for [('730a9b11-4347-4415-adec-96bd4e59e83e', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:35:25.354285 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2f91d5c3-89a3-4e2e-b1c5-5e41a8761843 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:25.394193 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2f91d5c3-89a3-4e2e-b1c5-5e41a8761843 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1214 microversion: 2.1 time: 0.670656 Oct 04 08:35:25.395846 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 14/28] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:24 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1214 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:26.236605 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-35415671-0e23-4ad5-a21c-c0af51d9fd10 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: .version_select at 0x7f30a0eb2280>, body: {"keypair": {"name": "tempest-keypair-607435801", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM0YkLtaz9WjGX/zr9Gd19Qz5X1Iibhiz6/LjTzt8ZET6tgdR6iB+PGBcEc8AqR7nQr8ya7rLFJPWX4/Vxq1d0iiCauR6xMnbDA8LDFdkvPI3ERYChLr4nhW43OWNB1JOw=="}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:26.329191 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-35415671-0e23-4ad5-a21c-c0af51d9fd10 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:35:26.353460 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-35415671-0e23-4ad5-a21c-c0af51d9fd10 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:35:26.512381 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-35415671-0e23-4ad5-a21c-c0af51d9fd10 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 1.114414 Oct 04 08:35:26.530123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 15/29] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:35:25 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 1133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:26.539922 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-85ecdb0a-6fb9-4202-b14c-0ebe2042523d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:26.543001 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-85ecdb0a-6fb9-4202-b14c-0ebe2042523d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:26.756255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-85ecdb0a-6fb9-4202-b14c-0ebe2042523d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:26.776117 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-85ecdb0a-6fb9-4202-b14c-0ebe2042523d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1214 microversion: 2.1 time: 0.244340 Oct 04 08:35:26.777577 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 16/30] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:26 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1214 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:26.818569 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=4 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:27.805154 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:27.805739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:27.812950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:27.813443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:27.813885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:27.826965 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:27.827673 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:27.828210 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:28.273335 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:28.292204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3cbfc8b7-a446-4d5a-8d5e-979716a68d60 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1210 microversion: 2.1 time: 0.490847 Oct 04 08:35:28.293502 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 17/31] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:27 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1210 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:28.831713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:35:28.832283 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:35:28.832608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:35:28.833022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:35:28.833539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:35:28.860946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:28.866368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1593225411", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:28.867591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:28.868052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:28.869009 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:28.869477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:28.869947 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:28.953620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for project 3e938efbd6804a86946d80ae92e88b16. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:35:28.969202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for user 87ca9fc2e013477cad3a3de8e8811274 and project 3e938efbd6804a86946d80ae92e88b16. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:35:29.025534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:35:29.028460 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:35:29.042255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] validate_networks() for [('7c0853df-a74a-47d5-9683-581a895763eb', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:35:29.234659 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e4e7a860-a34b-4874-9ba7-1eaa695ff469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 4.866357 Oct 04 08:35:29.236084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 15/32] 10.222.0.16 () {62 vars in 1188 bytes} [Tue Oct 4 08:35:24 2022] POST /compute/v2.1/servers => generated 402 bytes in 4869 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Oct 04 08:35:29.250728 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-61ad175b-80a3-4173-bb25-6d8d9f56d6b7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:29.251641 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-61ad175b-80a3-4173-bb25-6d8d9f56d6b7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:29.467682 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-61ad175b-80a3-4173-bb25-6d8d9f56d6b7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:29.510678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-61ad175b-80a3-4173-bb25-6d8d9f56d6b7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:29.512835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-61ad175b-80a3-4173-bb25-6d8d9f56d6b7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:29.514192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-61ad175b-80a3-4173-bb25-6d8d9f56d6b7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1280 microversion: 2.20 time: 0.269310 Oct 04 08:35:29.515464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 16/33] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:29 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1280 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:29.521384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:29.526360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:29.556112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:29.556632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:29.557080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:29.563705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:29.565930 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:29.566389 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:30.050467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:30.085911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c6bbd905-3689-4f7b-a3db-25a1b8e40aa9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1272 microversion: 2.1 time: 0.569173 Oct 04 08:35:30.086884 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 17/34] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:29 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1272 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:30.546293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-eac3c1ba-4d99-4313-94b6-246d6d8ff619 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:30.546972 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-eac3c1ba-4d99-4313-94b6-246d6d8ff619 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:31.080300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-eac3c1ba-4d99-4313-94b6-246d6d8ff619 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:31.091078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-eac3c1ba-4d99-4313-94b6-246d6d8ff619 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:31.092571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-eac3c1ba-4d99-4313-94b6-246d6d8ff619 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:31.101465 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-eac3c1ba-4d99-4313-94b6-246d6d8ff619 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1280 microversion: 2.20 time: 0.556205 Oct 04 08:35:31.102304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 18/35] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:30 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1280 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:35:31.124972 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:31.132868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:31.167237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:31.169201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:31.171023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:31.173682 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:31.175720 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:31.180362 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:31.515450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:31.539082 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f9813d32-8828-49d8-813e-3b4aa14c9a73 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1272 microversion: 2.1 time: 0.426680 Oct 04 08:35:31.540615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 19/36] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:31 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1272 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:31.656259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1730917701", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd380db5-2756-484c-8401-ea63df26e43d"}], "security_groups": [{"name": "tempest-securitygroup--256096983"}], "key_name": "tempest-keypair-607435801", "user_data": "}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:32.951485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] validate_networks() for [('fd380db5-2756-484c-8401-ea63df26e43d', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:35:34.308834 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:34.379697 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:35:34.380203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:35:34.388879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:35:34.389592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:35:34.390049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:35:34.417766 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:34.418299 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:34.418793 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:34.429389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:34.431762 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:34.432258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:34.493393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for project 5934621267af4b838889fcbfb9febaea. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:35:34.524210 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for user 26abb94392364bdc82f6bb508a611d44 and project 5934621267af4b838889fcbfb9febaea. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:35:34.565174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:35:34.589062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:35:34.772396 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a7bcd31a-740c-4670-b109-d4b8a66cc057 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 6.475581 Oct 04 08:35:34.774625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:35:28 2022] POST /compute/v2.1/servers => generated 383 bytes in 6479 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:35:34.792627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:34.793834 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:34.811563 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:34.812486 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:34.815408 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:34.830109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:34.830109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:34.830109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:35.501687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:35.516004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:35.518357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:35.525046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8c073e96-763d-4c4a-8a70-f0b079d0e79c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1338 microversion: 2.20 time: 0.747497 Oct 04 08:35:35.525900 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 19/38] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:34 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1338 bytes in 751 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:35.535018 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:35.537248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:35.565059 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:35.568785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:35.568785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:35.571549 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:35.576620 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:35.577481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:37.013008 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:37.028486 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-443dbe74-60e2-4b55-979b-94dc6cb8f24f tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 1.500817 Oct 04 08:35:37.052058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 20/39] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:35 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 1525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:37.067379 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:37.071011 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:37.094348 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:37.096954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:37.097620 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:37.101169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:37.103075 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:37.105180 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:38.369518 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:38.385014 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-becdf190-b3cc-446b-bb49-a1462da9e2ee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1274 microversion: 2.1 time: 1.325962 Oct 04 08:35:38.385907 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 21/40] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:37 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1274 bytes in 1333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:38.393403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:38.398783 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:38.413413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:38.415875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:38.431391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:38.432441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:38.433056 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:38.433648 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:38.683868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:35:38.683868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:35:38.683868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:35:38.683868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:35:38.683868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:35:38.683868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=4 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:38.704593 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:38.705092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:38.705539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:38.708756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:38.709429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:38.712161 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:38.726004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:38.749397 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:38.754909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:38.760226 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cbd3ff58-0690-4a70-841e-08acad161b07 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1338 microversion: 2.20 time: 0.370603 Oct 04 08:35:38.760831 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 22/41] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:38 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1338 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:38.817590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:38.818374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:38.826045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:35:38.834265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:38.835878 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:35:38.837921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:38.839027 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:38.842242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:38.843900 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:38.844537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:38.867630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:35:38.877688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:35:38.946838 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-aab973c8-3355-490e-b184-24e6aab9c3fa tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 7.300170 Oct 04 08:35:38.955214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 20/42] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:35:31 2022] POST /compute/v2.1/servers => generated 401 bytes in 7311 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:35:38.964791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2052287189", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:39.167734 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] validate_networks() for [('7c0853df-a74a-47d5-9683-581a895763eb', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:35:39.513341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:39.565741 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-62646c7e-9ffe-4994-9248-99c9be3a0e98 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.794728 Oct 04 08:35:39.573838 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 23/43] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:38 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:39.591978 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f2e03a16-45a1-47cf-960a-b096ff97f4fb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:39.593091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-f2e03a16-45a1-47cf-960a-b096ff97f4fb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:39.749639 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f2e03a16-45a1-47cf-960a-b096ff97f4fb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:39.764914 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f2e03a16-45a1-47cf-960a-b096ff97f4fb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1241 microversion: 2.1 time: 0.187500 Oct 04 08:35:39.780989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 24/44] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:39 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1241 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:39.797645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:39.800199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:39.817592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:39.819292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:39.830686 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:39.833062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:39.835468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:39.836138 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:42.923166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:42.944094 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:42.945608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:42.946793 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4a01f23f-480f-4974-ae34-7792e9ccae54 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 3.160070 Oct 04 08:35:42.947525 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 25/45] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:39 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 3165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:42.956523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:42.959525 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:42.976890 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:42.980723 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:42.981189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:42.987008 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:42.988967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:42.990011 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:43.365980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:35:43.368247 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:35:43.368745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:35:43.369267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:35:43.369695 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:35:43.389551 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:43.390147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:43.390609 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:43.391335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:43.391858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:43.392418 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:43.438150 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for project 5934621267af4b838889fcbfb9febaea. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:35:43.446890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for user 26abb94392364bdc82f6bb508a611d44 and project 5934621267af4b838889fcbfb9febaea. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:35:43.475421 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:35:43.486883 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:35:43.541454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:43.555162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6cbb18e9-0412-4312-b0e0-83c58cd8737f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.598458 Oct 04 08:35:43.556346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 21/46] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:35:38 2022] POST /compute/v2.1/servers => generated 383 bytes in 4600 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:35:43.563227 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:43.563866 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:43.569612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4ed96eaa-1598-4f66-9b6e-dbe5130bf8e9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.620717 Oct 04 08:35:43.571699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 26/47] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:42 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:43.572190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:43.572946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:43.573447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:43.574214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:43.574805 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:43.575304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:43.735483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:43.751177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-286eff23-5679-426c-9ff2-cf3554118d09 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1299 microversion: 2.1 time: 0.193454 Oct 04 08:35:43.762985 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 22/48] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:43 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1299 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:43.788835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-72c3c90c-f182-4bc7-ace0-af0ef8d66e7a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:43.794960 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-72c3c90c-f182-4bc7-ace0-af0ef8d66e7a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:43.897649 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-72c3c90c-f182-4bc7-ace0-af0ef8d66e7a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:43.907567 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-72c3c90c-f182-4bc7-ace0-af0ef8d66e7a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1222 microversion: 2.1 time: 0.142940 Oct 04 08:35:43.909697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 23/49] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:43 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1222 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:43.947200 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-f3f3db65-b52b-404b-aa2a-c86e8a67edaf req-cd1c7a6a-7d26-40d2-9907-6d347a576cf0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7bf8ac1b-807c-4209-aab2-2d6aff4038f0", "tag": "52f10973-3a78-447b-a7be-d1bc8b3c7e32"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:43.979620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:43.985365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:43.986572 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f3f3db65-b52b-404b-aa2a-c86e8a67edaf req-cd1c7a6a-7d26-40d2-9907-6d347a576cf0 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:43.987224 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f3f3db65-b52b-404b-aa2a-c86e8a67edaf req-cd1c7a6a-7d26-40d2-9907-6d347a576cf0 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:43.987532 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f3f3db65-b52b-404b-aa2a-c86e8a67edaf req-cd1c7a6a-7d26-40d2-9907-6d347a576cf0 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:43.999370 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.000160 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.000672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.001490 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.001964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.002383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.050288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-f3f3db65-b52b-404b-aa2a-c86e8a67edaf req-cd1c7a6a-7d26-40d2-9907-6d347a576cf0 service nova] Creating event network-changed:52f10973-3a78-447b-a7be-d1bc8b3c7e32 for instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 on hv-cinder-86020 Oct 04 08:35:44.050873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f3f3db65-b52b-404b-aa2a-c86e8a67edaf req-cd1c7a6a-7d26-40d2-9907-6d347a576cf0 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127142) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 04 08:35:44.051278 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f3f3db65-b52b-404b-aa2a-c86e8a67edaf req-cd1c7a6a-7d26-40d2-9907-6d347a576cf0 service nova] Acquired lock "compute-rpcapi-router" {{(pid=127142) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 04 08:35:44.052875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.053341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.053771 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.054494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.062826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.063555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.092052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.compute.rpcapi [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 04 08:35:44.092957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] Releasing lock "compute-rpcapi-router" {{(pid=127142) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 04 08:35:44.110056 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-962c6f03-7bf7-4096-8023-be904dd4c0cd None None] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.535314 Oct 04 08:35:44.111682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 27/50] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:35:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:44.283112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:44.293674 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:44.295531 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:44.296671 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-dde0af92-b609-4b2f-923d-8286ea20f58e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.329135 Oct 04 08:35:44.298903 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 24/51] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:43 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:44.595491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:44.596192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:44.605119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.605642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.606059 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.606797 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.607244 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.607660 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.780523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:44.789513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-edd2c2ef-7c6e-4b79-98f3-4be136bfc798 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.198273 Oct 04 08:35:44.790664 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 28/52] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:44 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:44.791507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:44.794146 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:44.811085 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.812141 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.813032 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.814390 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.815337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.816169 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.884929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:44.893099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-30e82302-4bf1-474d-9d10-239f2d39d1e0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1299 microversion: 2.1 time: 0.111539 Oct 04 08:35:44.893684 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 25/53] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:44 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1299 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:44.938187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:44.939337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:44.950329 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.951252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.952023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:44.953274 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:44.954108 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:44.954924 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:45.028044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:45.036986 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3d415ceb-037e-4000-836f-fbe7792427fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1226 microversion: 2.1 time: 0.105891 Oct 04 08:35:45.037587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 29/54] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:44 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1226 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:45.325081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:45.325081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:45.331684 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:45.332172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:45.332603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:45.333955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:45.334408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:45.334872 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:45.476756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:45.485203 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:45.486670 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:45.487773 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-39b5b243-a813-4506-9915-085444575691 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.167711 Oct 04 08:35:45.488360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 26/55] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:45 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:45.816189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:45.817048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:45.824682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:45.825657 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:45.830996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:45.835749 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:45.837036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:45.837473 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:45.918204 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:45.919557 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:45.941924 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:45.943360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:45.943811 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:45.944540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:45.944999 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:45.945425 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:46.050658 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:46.059472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d43d3ac8-8fd9-43c8-af15-4c2711ba9278 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1299 microversion: 2.1 time: 0.145009 Oct 04 08:35:46.062609 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:46.070013 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 27/56] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:45 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1299 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:46.076991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c31976ff-68b0-47e3-a089-4946f8830810 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.266897 Oct 04 08:35:46.077593 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 30/57] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:45 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:46.105850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:46.106528 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:46.115670 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:46.116535 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:46.117172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:46.118102 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:46.119011 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:46.119617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:46.132157 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-05d04457-830c-4d71-aac4-91c466911005 req-4e17e20b-ed85-43c2-a697-2283bdd60c25 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54819895-e219-4341-8f1e-5543b7357315", "tag": "159277a9-e8de-47c4-813b-8de31279c18f"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:46.169331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-05d04457-830c-4d71-aac4-91c466911005 req-4e17e20b-ed85-43c2-a697-2283bdd60c25 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:46.170033 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-05d04457-830c-4d71-aac4-91c466911005 req-4e17e20b-ed85-43c2-a697-2283bdd60c25 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:46.170670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-05d04457-830c-4d71-aac4-91c466911005 req-4e17e20b-ed85-43c2-a697-2283bdd60c25 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:46.190847 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-05d04457-830c-4d71-aac4-91c466911005 req-4e17e20b-ed85-43c2-a697-2283bdd60c25 service nova] Creating event network-changed:159277a9-e8de-47c4-813b-8de31279c18f for instance 54819895-e219-4341-8f1e-5543b7357315 on hv-cinder-86020 Oct 04 08:35:46.197879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-05d04457-830c-4d71-aac4-91c466911005 req-4e17e20b-ed85-43c2-a697-2283bdd60c25 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069285 Oct 04 08:35:46.199091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 31/58] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:35:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:46.219090 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:46.227052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-86e28c9d-8559-4d66-aee6-6ba041ca14fc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1226 microversion: 2.1 time: 0.150480 Oct 04 08:35:46.227839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 28/59] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:46 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1226 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:46.517527 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:46.518968 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:46.529573 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:46.530175 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:46.530706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:46.531487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:46.532000 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:46.532539 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:46.721254 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:46.731860 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:46.733623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:46.734781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-35b01dc0-ee3c-4a64-95e8-030b30bc6e3c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.222155 Oct 04 08:35:46.735491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 32/60] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:46 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:47.094415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:47.096768 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:47.103161 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:47.104078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.104571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.104998 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.105468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:47.105684 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.106296 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.106946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.113611 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.114865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.115575 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.116672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.117338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.117943 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.218337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:47.227005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2b560e5d-1606-4007-8f91-026093339ab9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1299 microversion: 2.1 time: 0.137141 Oct 04 08:35:47.227808 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 29/61] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:47 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1299 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:47.251801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:47.252587 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:47.263052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.263561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.264002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.264716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.265199 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.265581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.302059 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:47.313546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cdfa4d3f-7350-4952-8930-5833e0c1b01e tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.215369 Oct 04 08:35:47.328055 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 33/62] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:47 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:47.347961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:47.356012 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8759fa24-91c7-4f6c-8a7c-3414a96ab07f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1226 microversion: 2.1 time: 0.108856 Oct 04 08:35:47.358581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 30/63] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:47 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1226 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:47.762678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:47.763295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:47.765224 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=7 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:47.771032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.771584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.772015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.772706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:47.773150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:47.773574 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:47.778375 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=89 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:47.937032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:47.948223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:47.950995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:47.953406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c04e3b1f-4ec7-460c-a827-e899f865502d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.194701 Oct 04 08:35:47.956828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 34/64] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:47 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:48.250865 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:48.251644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:48.257865 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.258336 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.258789 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:48.259468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.259905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.260317 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:48.323711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:48.332464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-84cd034f-453d-4ae3-a4b2-d81296f4507e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1299 microversion: 2.1 time: 0.085721 Oct 04 08:35:48.333334 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 31/65] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:48 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1299 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:48.337662 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:48.338672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:48.344374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.344861 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.345299 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:48.345986 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.346436 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.347045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:48.380296 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:48.380587 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:48.386817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.387305 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.387910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:48.388614 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.389067 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.389494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:48.472226 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:48.480783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-78cd5bed-9c7e-440f-bdea-791f29ec63dc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1226 microversion: 2.1 time: 0.104551 Oct 04 08:35:48.481365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 32/66] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:48 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1226 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:48.509725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:48.517616 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a2078edd-b39a-44cc-8749-f5e2a504d576 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.183794 Oct 04 08:35:48.518189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 35/67] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:48 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:48.983646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:48.984124 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:48.993655 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.994834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.995344 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:48.996567 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:48.997114 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:48.997793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:49.356134 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:49.357488 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:49.365917 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:49.366845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:49.367507 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:49.368477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:49.369142 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:49.369798 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:49.675298 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:49.690100 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4ca9d49d-a9aa-4f23-853c-5d9bcbb3fd36 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1299 microversion: 2.1 time: 0.338072 Oct 04 08:35:49.691773 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 36/68] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:49 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1299 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:49.706596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:49.707904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:49.726955 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:49.728242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:49.728943 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:49.730389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:49.731150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:49.731734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:49.768553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=22 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:49.862748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:49.865633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=83 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:49.880745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:49.888210 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:49.890288 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6a1296ff-8e1e-47aa-be13-825710c6da86 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.912229 Oct 04 08:35:49.892866 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 33/69] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:48 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:49.914350 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:49.915018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:49.936203 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:49.939283 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:49.940408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:49.941371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:49.942173 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:49.946443 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:50.215459 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:50.250956 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ff6ce183-a6ac-41a8-ac3a-d37be9de2245 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1274 microversion: 2.1 time: 0.544353 Oct 04 08:35:50.252044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 37/70] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:49 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1274 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:50.434868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:50.444227 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f771f22d-952a-40b7-8cba-3fd1095a96a9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.548890 Oct 04 08:35:50.449597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 34/71] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:49 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:50.718474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:50.719297 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:50.732289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:50.732989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:50.734242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:50.737190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:50.737880 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:50.739047 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:50.834016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-0de633df-ad07-4a2d-864a-21546f29eaa4 req-4829973c-c4fe-403c-b3d6-5c1e0bec714d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f597e06f-510c-46fe-92de-ce4a8dc53841", "tag": "0e360ecf-b764-45c6-8c4f-8ea15e619e9e"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:50.875809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0de633df-ad07-4a2d-864a-21546f29eaa4 req-4829973c-c4fe-403c-b3d6-5c1e0bec714d service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:50.876331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0de633df-ad07-4a2d-864a-21546f29eaa4 req-4829973c-c4fe-403c-b3d6-5c1e0bec714d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:50.876772 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0de633df-ad07-4a2d-864a-21546f29eaa4 req-4829973c-c4fe-403c-b3d6-5c1e0bec714d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:50.921537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-0de633df-ad07-4a2d-864a-21546f29eaa4 req-4829973c-c4fe-403c-b3d6-5c1e0bec714d service nova] Creating event network-changed:0e360ecf-b764-45c6-8c4f-8ea15e619e9e for instance f597e06f-510c-46fe-92de-ce4a8dc53841 on hv-cinder-86020 Oct 04 08:35:50.939842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-0de633df-ad07-4a2d-864a-21546f29eaa4 req-4829973c-c4fe-403c-b3d6-5c1e0bec714d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.284996 Oct 04 08:35:50.944406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 38/72] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:35:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:50.962920 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:50.966121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:50.978174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:50.986191 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:50.988630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:50.992917 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:50.993755 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:50.995195 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:51.091732 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:51.103450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-286c5086-9d38-4649-911a-b1b945a563c1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1299 microversion: 2.1 time: 0.389577 Oct 04 08:35:51.105686 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 35/73] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:50 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1299 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:51.277275 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:51.277794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:51.291407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:51.292194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:51.292698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:51.293489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:51.295042 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:51.295797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:51.460273 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:51.470303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-006d2451-f98d-42fd-ba4e-91c0fa734f54 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1280 microversion: 2.1 time: 0.196790 Oct 04 08:35:51.471584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 36/74] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:51 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1280 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:51.480282 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:51.480838 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:51.487577 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:51.488125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:51.488588 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:51.489957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:51.490434 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:51.490901 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:51.711030 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:51.720728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:51.722278 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:51.723590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9807b2c1-de63-41d3-ad82-a12b178058e4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.777628 Oct 04 08:35:51.724376 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 39/75] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:50 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:52.040986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:52.078311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f7208d1b-5632-48a0-99e9-d07b6e9bc286 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.601782 Oct 04 08:35:52.080152 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 37/76] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:51 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:52.143864 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:52.144778 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:52.154439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:52.155040 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:52.155551 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:52.156364 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:52.156933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:52.157436 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:52.502745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:52.502745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:52.519270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:52.521518 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:52.522176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:52.523916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:52.524636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:52.527352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:53.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=6 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:53.493952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:53.505762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:53.525554 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1979c624-5d9a-4169-8f66-f47fb99701a8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 1.032551 Oct 04 08:35:53.533233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0d65aa25-a83a-4c80-918b-0cc5c5eda4af tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 1.404847 Oct 04 08:35:53.556378 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 38/77] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:52 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 1064 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:53.571793 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 40/78] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:52 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 1444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:53.589346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:53.589931 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:53.597692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:53.598174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:53.599300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:53.599975 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:53.600659 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:53.601383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:53.610357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:53.622688 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:53.647072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:53.652720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:53.653372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:53.655562 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:53.656315 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:53.658572 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:53.903030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=24 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:54.092123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:54.763701 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:54.790295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:54.798348 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:54.799596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-bcc04e05-9dec-433a-9ce9-ed2c70a20906 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 1.226155 Oct 04 08:35:54.800198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 39/79] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:53 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 1243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:54.825209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:54.826272 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:54.849445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:54.851888 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:54.852826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:54.855288 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:54.857425 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:54.858625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:55.038203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:55.059986 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-257cfc92-19cf-4dd4-9dc8-8e64fefa7ba5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 1.479976 Oct 04 08:35:55.064319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 41/80] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:53 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 1492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:55.074314 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:55.074966 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:55.093721 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:55.094234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:55.094712 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:55.095447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:55.095899 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:55.097797 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:56.044357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:56.055864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-256bba5b-f47d-4be6-9a66-dc4ffd5fe053 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 1.254551 Oct 04 08:35:56.057467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 40/81] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:54 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 1257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:56.064392 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:56.066144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:56.085797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:56.090055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:56.090722 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:56.091510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:56.091995 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:56.092426 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:56.182450 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:56.193958 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-99ad6383-93e1-414f-b52e-3ea4adac9d2e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 1.128328 Oct 04 08:35:56.211625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 42/82] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:55 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 1147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:56.217021 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:56.218664 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:56.227072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:56.227523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:56.227999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:56.228673 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:56.229288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:56.229715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:56.648403 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:56.659277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:56.664061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:56.669427 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-45ac7643-fe28-498a-8760-63554c229372 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.609403 Oct 04 08:35:56.670137 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 41/83] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:56 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:56.720723 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:56.730461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b1377517-2e95-40be-941e-d434d29d36b4 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.517448 Oct 04 08:35:56.734645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 43/84] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:56 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:57.096966 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:57.099078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:57.124162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.124745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-2c874ef6-97b7-4b8a-818c-7e9508bcb259 req-fab847f2-49f0-4e9a-9cb9-985acca649d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9fd4027-9ae5-4ab2-afc6-c7eb723204cf", "tag": "364e2511-1153-48e7-8595-032dbdba4297"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:57.125271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.125926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.126916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.127631 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.128263 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.161656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2c874ef6-97b7-4b8a-818c-7e9508bcb259 req-fab847f2-49f0-4e9a-9cb9-985acca649d8 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.161656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2c874ef6-97b7-4b8a-818c-7e9508bcb259 req-fab847f2-49f0-4e9a-9cb9-985acca649d8 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.162013 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2c874ef6-97b7-4b8a-818c-7e9508bcb259 req-fab847f2-49f0-4e9a-9cb9-985acca649d8 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.198726 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-2c874ef6-97b7-4b8a-818c-7e9508bcb259 req-fab847f2-49f0-4e9a-9cb9-985acca649d8 service nova] Creating event network-changed:364e2511-1153-48e7-8595-032dbdba4297 for instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf on hv-cinder-86020 Oct 04 08:35:57.206599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-2c874ef6-97b7-4b8a-818c-7e9508bcb259 req-fab847f2-49f0-4e9a-9cb9-985acca649d8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086796 Oct 04 08:35:57.209048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 44/85] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:35:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:35:57.233015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:57.234501 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:57.243972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.244777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.245328 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.246315 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.247200 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.248038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.341464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:57.354367 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e5d40fd2-13ee-494f-8941-8b443cb531a5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.259085 Oct 04 08:35:57.355307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 42/86] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:57 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:57.462839 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:57.470999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2a0df64d-110f-4ef9-ab12-a8b3dcca980f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.251711 Oct 04 08:35:57.471595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 45/87] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:57 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:57.698885 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:57.699410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:57.706101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.706616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.707053 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.708293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.708743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.709286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.759063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:57.759803 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:57.766098 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.766614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.767066 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.767777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:57.768233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:57.768659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:57.884607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:57.893670 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:57.895749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:57.897031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1889b78f-4d50-49cb-9480-51bca53ee914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.202157 Oct 04 08:35:57.897620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 43/88] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:57 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:57.933643 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:57.941494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-afda14d4-15cf-48f8-b82d-14ac61aaa508 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.186470 Oct 04 08:35:57.942599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 46/89] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:57 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:58.377822 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:58.378472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:58.385371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:58.385843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:58.386269 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:58.386994 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:58.387450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:58.387870 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:58.493640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:58.494330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:58.501374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:58.501976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:58.502508 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:58.503466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:58.504236 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:58.505042 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:58.534777 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:58.544696 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3f0d05bc-869d-4055-aa77-eb86fbc970f0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.170959 Oct 04 08:35:58.545504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 44/90] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:58 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:35:58.932156 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:58.936372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:58.955437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:58.956242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:58.956858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:58.963553 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:58.964568 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:58.968038 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:59.344048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:59.355841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f59776b6-7063-4bcb-8745-960d1529626d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.865676 Oct 04 08:35:59.372702 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 47/91] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:58 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 884 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:35:59.393248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:35:59.395346 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:35:59.410170 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:59.412912 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:59.414440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:59.427873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:59.428604 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:59.429177 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:59.833918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:59.867634 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:59.871260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:35:59.876254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d94b04d5-8c1a-4ed3-a256-dd31b8807473 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.957363 Oct 04 08:35:59.878783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 45/92] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:35:58 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 962 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:35:59.885824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-137a60cf-79fa-43e8-a49b-6750537da4a8 req-bdae0fd3-3cdd-4462-be50-0aaec616cf1c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8351183-2123-48d0-9f68-c06ab2829d71", "tag": "4b6d91d8-8d57-437a-8e26-84b89843a14e"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:35:59.914859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-137a60cf-79fa-43e8-a49b-6750537da4a8 req-bdae0fd3-3cdd-4462-be50-0aaec616cf1c service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:59.915766 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-137a60cf-79fa-43e8-a49b-6750537da4a8 req-bdae0fd3-3cdd-4462-be50-0aaec616cf1c service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:59.916430 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-137a60cf-79fa-43e8-a49b-6750537da4a8 req-bdae0fd3-3cdd-4462-be50-0aaec616cf1c service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:59.955349 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-137a60cf-79fa-43e8-a49b-6750537da4a8 req-bdae0fd3-3cdd-4462-be50-0aaec616cf1c service nova] Creating event network-changed:4b6d91d8-8d57-437a-8e26-84b89843a14e for instance a8351183-2123-48d0-9f68-c06ab2829d71 on hv-cinder-86020 Oct 04 08:35:59.956476 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-137a60cf-79fa-43e8-a49b-6750537da4a8 req-bdae0fd3-3cdd-4462-be50-0aaec616cf1c service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127143) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 04 08:35:59.957340 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-137a60cf-79fa-43e8-a49b-6750537da4a8 req-bdae0fd3-3cdd-4462-be50-0aaec616cf1c service nova] Acquired lock "compute-rpcapi-router" {{(pid=127143) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 04 08:35:59.959946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:59.960801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:59.962608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:59.963984 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:35:59.964806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:35:59.965421 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:35:59.983732 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.compute.rpcapi [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 04 08:35:59.984965 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] Releasing lock "compute-rpcapi-router" {{(pid=127143) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 04 08:36:00.017320 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6d46e7de-bf47-473a-b53d-bb071bd3d645 None None] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.136535 Oct 04 08:36:00.018257 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 46/93] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:35:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:36:00.058128 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:00.069900 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0e1b54be-2f7b-44b7-a207-ac1af6c39d8b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.694911 Oct 04 08:36:00.070596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 48/94] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:35:59 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 697 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:00.104798 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:00.105867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:00.117658 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:00.118383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:00.119136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:00.120187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:00.121035 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:00.121667 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:00.173535 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-02de8139-5972-4c8b-8a09-8799ed29d17e req-6b9c9fac-0c46-4108-a693-871e4d9ee80d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7bf8ac1b-807c-4209-aab2-2d6aff4038f0", "name": "network-vif-plugged", "status": "completed", "tag": "52f10973-3a78-447b-a7be-d1bc8b3c7e32"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:00.194441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-02de8139-5972-4c8b-8a09-8799ed29d17e req-6b9c9fac-0c46-4108-a693-871e4d9ee80d service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:00.195036 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-02de8139-5972-4c8b-8a09-8799ed29d17e req-6b9c9fac-0c46-4108-a693-871e4d9ee80d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:00.195791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-02de8139-5972-4c8b-8a09-8799ed29d17e req-6b9c9fac-0c46-4108-a693-871e4d9ee80d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:00.212617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-02de8139-5972-4c8b-8a09-8799ed29d17e req-6b9c9fac-0c46-4108-a693-871e4d9ee80d service nova] Creating event network-vif-plugged:52f10973-3a78-447b-a7be-d1bc8b3c7e32 for instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 on hv-cinder-86020 Oct 04 08:36:00.219861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-02de8139-5972-4c8b-8a09-8799ed29d17e req-6b9c9fac-0c46-4108-a693-871e4d9ee80d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.199615 Oct 04 08:36:00.221039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 47/95] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:36:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:36:00.338333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:00.348259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c5d19de8-a340-4aeb-818a-aba347dbe18e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.269058 Oct 04 08:36:00.348945 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 49/96] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:00 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:00.387503 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:00.388248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:00.395766 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:00.396285 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:00.396750 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:00.397502 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:00.398092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:00.398528 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:00.601819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:00.610748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-07e64487-cac8-443f-bd2b-094894b64c05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.227303 Oct 04 08:36:00.611476 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 48/97] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:00 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:00.909804 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:00.910423 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:00.917761 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:00.918243 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:00.918707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:00.919406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:00.919953 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:00.920393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:01.076696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:01.087653 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:01.089683 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:01.091295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7cb36a19-14e5-4e76-a23a-ba5569cc2f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.185752 Oct 04 08:36:01.102578 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 50/98] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:36:00 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:36:01.102820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:01.102820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:01.104542 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:01.105100 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:01.105597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:01.106565 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:01.106984 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:01.107396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:01.272304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:01.280508 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-773c6873-5974-445d-ae23-5bca81b5b56c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.187664 Oct 04 08:36:01.281319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 49/99] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:01 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:01.371825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:01.372477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:01.379137 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:01.380467 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:01.380967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:01.381782 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:01.382311 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:01.382977 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:01.537891 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:01.545619 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-53311738-23a4-4d8d-8f3c-55df75da546a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.177778 Oct 04 08:36:01.546203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 51/100] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:01 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:01.638290 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:01.638952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:01.645476 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:01.645990 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:01.646455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:01.647191 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:01.647640 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:01.648204 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:01.799235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:01.812354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fc6fb7c8-3cb4-474e-85d6-f67e9ef0165d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.178931 Oct 04 08:36:01.813005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 50/101] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:01 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:02.120492 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:02.121760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:02.128678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.129210 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.130274 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.131341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.131791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.132222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.136579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:02.237643 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-fbd38e5a-af24-4551-961b-ee83bc1b9c2f req-bcbc8377-e413-4040-b78f-280e6fdf96d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f597e06f-510c-46fe-92de-ce4a8dc53841", "name": "network-vif-plugged", "status": "completed", "tag": "0e360ecf-b764-45c6-8c4f-8ea15e619e9e"}, {"server_uuid": "54819895-e219-4341-8f1e-5543b7357315", "name": "network-vif-plugged", "status": "completed", "tag": "159277a9-e8de-47c4-813b-8de31279c18f"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:02.249205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-fbd38e5a-af24-4551-961b-ee83bc1b9c2f req-bcbc8377-e413-4040-b78f-280e6fdf96d6 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.250029 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-fbd38e5a-af24-4551-961b-ee83bc1b9c2f req-bcbc8377-e413-4040-b78f-280e6fdf96d6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.250173 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-fbd38e5a-af24-4551-961b-ee83bc1b9c2f req-bcbc8377-e413-4040-b78f-280e6fdf96d6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.265798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-fbd38e5a-af24-4551-961b-ee83bc1b9c2f req-bcbc8377-e413-4040-b78f-280e6fdf96d6 service nova] Creating event network-vif-plugged:0e360ecf-b764-45c6-8c4f-8ea15e619e9e for instance f597e06f-510c-46fe-92de-ce4a8dc53841 on hv-cinder-86020 Oct 04 08:36:02.266302 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-fbd38e5a-af24-4551-961b-ee83bc1b9c2f req-bcbc8377-e413-4040-b78f-280e6fdf96d6 service nova] Creating event network-vif-plugged:159277a9-e8de-47c4-813b-8de31279c18f for instance 54819895-e219-4341-8f1e-5543b7357315 on hv-cinder-86020 Oct 04 08:36:02.274795 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-fbd38e5a-af24-4551-961b-ee83bc1b9c2f req-bcbc8377-e413-4040-b78f-280e6fdf96d6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.041508 Oct 04 08:36:02.281280 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:02.284836 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 51/102] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:36:02 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:36:02.290014 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:02.291493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:02.292803 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6cb84a67-ba53-4477-8c65-32d19d05697d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.178872 Oct 04 08:36:02.295121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 52/103] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:36:02 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:36:02.312570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:02.313959 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:02.325923 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.326541 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.326996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.327687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.328130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.328549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.331965 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:02.468879 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:02.477512 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4d7b3e4c-3704-46bc-a1b0-98a00cbee052 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.174112 Oct 04 08:36:02.478092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 52/104] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:02 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:02.571330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:02.572289 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:02.579865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.580851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.581328 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.582038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.583015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.583926 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.839284 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:02.839905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:02.851984 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.852955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.853613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:02.855057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:02.855777 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:02.856450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:03.201686 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:03.214079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6104d1cf-e8e0-4fbb-ba8f-2d6349d7993e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.648080 Oct 04 08:36:03.215197 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 53/105] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:02 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:03.247523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:03.258126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b0e7022f-1953-4610-b126-a7813e068939 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.424111 Oct 04 08:36:03.258846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 53/106] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:02 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:03.323290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:03.323909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:03.330986 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:03.331467 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:03.331987 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:03.332613 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:03.333146 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:03.333454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:03.503079 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:03.503079 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:03.511856 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:03.513510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:03.513998 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:03.514761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:03.519187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:03.519725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:03.526273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:03.537627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:03.540690 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:03.543191 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a6debd54-e2ae-4b64-9bae-38029f0fedf3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.225589 Oct 04 08:36:03.544354 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 54/107] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:36:03 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:36:03.731945 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:03.746449 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c81a153c-330a-4de4-b113-898c629aca40 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.250047 Oct 04 08:36:03.747278 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 54/108] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:03 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:04.236652 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:04.237929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:04.244221 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.244704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.245130 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:04.245984 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.246609 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.247028 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:04.280779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:04.281409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:04.288116 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.288602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.289035 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:04.289724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.290206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.290608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:04.857341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:04.864922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:04.882936 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-496c9481-d93d-4e6c-90cb-68de639bccca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.649884 Oct 04 08:36:04.884367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 55/109] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:04 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:04.895070 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-26f724ee-9d8d-4ff5-bc60-24eeb94447f4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.608242 Oct 04 08:36:04.895070 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 55/110] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:04 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:04.897388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:04.898188 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:04.905953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:04.909707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:04.910321 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:04.911116 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.911760 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.912206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:04.913181 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.913644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.914978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:04.929264 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.929841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.931011 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:04.932332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:04.934830 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:04.935393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:05.420586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:05.429235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:05.430720 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:05.437483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9497c04a-c8f6-45c2-a654-d02817d0a2f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1406 microversion: 2.20 time: 0.549681 Oct 04 08:36:05.439035 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:05.441080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 56/111] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:36:04 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1406 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:36:05.452871 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9d577152-07ff-4e19-941b-041522d82121 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1314 microversion: 2.1 time: 0.549004 Oct 04 08:36:05.453807 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 56/112] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:04 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1314 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:05.911437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:05.912521 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:05.920761 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:05.921387 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:05.923831 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:05.930440 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:05.931327 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:05.931822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:05.932257 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:05.932957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:05.933359 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:05.934183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:05.935096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:05.936691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:05.936986 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:05.943618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:06.346806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:06.361127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:06.372771 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fd613732-d6ea-49f5-9853-731a0074018a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.465832 Oct 04 08:36:06.374716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 57/113] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:05 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:06.375915 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6c9d51a3-59b1-4a43-92f6-fff825527902 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.459940 Oct 04 08:36:06.392358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 57/114] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:05 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:06.470467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:06.471849 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:06.482802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:06.483893 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:06.484501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:06.485316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:06.485933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:06.486313 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:06.487014 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:06.487733 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:06.497383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:06.497641 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:06.503399 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:06.507037 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:06.507723 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:06.508325 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:06.764628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:06.776947 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:06.786791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:06.790407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-12de9ceb-f925-44b9-82cc-12b49e73ebad tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1599 microversion: 2.20 time: 0.323147 Oct 04 08:36:06.792575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 58/115] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:36:06 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1599 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:36:06.823304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:06.824041 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:06.838407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:06.840727 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:06.841693 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:06.842819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:06.843464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:06.844065 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:06.874030 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:06.884546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7f500e02-260f-4bbd-8723-dbde04235524 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1501 microversion: 2.1 time: 0.402503 Oct 04 08:36:06.885193 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 58/116] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:06 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1501 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:07.006961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e3540f44-1848-4fee-9f90-d0357714bd3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.191311 Oct 04 08:36:07.007660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 59/117] 10.222.0.16 () {60 vars in 1316 bytes} [Tue Oct 4 08:36:06 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-interface => generated 285 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:36:07.401713 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:07.402344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:07.403515 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:07.404131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:07.412674 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:07.412868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:07.413172 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:07.414444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:07.415168 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:07.418483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:07.418954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:07.419586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:07.419739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:07.420217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:07.421096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:07.421209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:07.952748 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:07.968825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0d12a4ac-1f5f-48f6-ad24-40d044621c0d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.572481 Oct 04 08:36:07.975691 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 59/118] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:07 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:08.086226 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-59bfdebc-7f6f-4fff-91cd-78f1cef0f495 req-c802ccad-6847-482a-b607-73d956bac420 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a8351183-2123-48d0-9f68-c06ab2829d71", "name": "network-vif-plugged", "status": "completed", "tag": "4b6d91d8-8d57-437a-8e26-84b89843a14e"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:08.107541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-59bfdebc-7f6f-4fff-91cd-78f1cef0f495 req-c802ccad-6847-482a-b607-73d956bac420 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:08.112391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-59bfdebc-7f6f-4fff-91cd-78f1cef0f495 req-c802ccad-6847-482a-b607-73d956bac420 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:08.116094 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-59bfdebc-7f6f-4fff-91cd-78f1cef0f495 req-c802ccad-6847-482a-b607-73d956bac420 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:08.167591 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-59bfdebc-7f6f-4fff-91cd-78f1cef0f495 req-c802ccad-6847-482a-b607-73d956bac420 service nova] Creating event network-vif-plugged:4b6d91d8-8d57-437a-8e26-84b89843a14e for instance a8351183-2123-48d0-9f68-c06ab2829d71 on hv-cinder-86020 Oct 04 08:36:08.179408 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-59bfdebc-7f6f-4fff-91cd-78f1cef0f495 req-c802ccad-6847-482a-b607-73d956bac420 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098003 Oct 04 08:36:08.182937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 60/119] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:36:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:36:08.406399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:08.416840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fc017534-46ac-4097-81e5-474ecd36354a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 1.017330 Oct 04 08:36:08.417879 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 60/120] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:07 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 1019 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:08.499885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5c14578b-6189-4c06-ba55-9af8b33ebf80"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:08.502978 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:08.515524 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:08.516243 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:08.516907 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:08.517907 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:08.518631 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:08.519283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:09.012495 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:09.015270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:09.033292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b7117464-1906-4316-b7ed-e3b71c5b30c9 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "POST /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.538156 Oct 04 08:36:09.034018 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 61/121] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:36:08 2022] POST /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0/os-volume_attachments => generated 194 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:36:09.044069 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:09.044564 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:09.047925 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:09.048719 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:09.051574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:09.052227 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:09.371942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:09.384600 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0a383198-4681-4a91-85ef-0be07b643091 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 0.394432 Oct 04 08:36:09.399437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 61/122] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:08 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:09.444067 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:09.444954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:09.464783 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:09.465879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:09.468559 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:09.470660 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:09.472119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:09.474926 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:09.905995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:09.925791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ab4c5cba-c8a6-436d-889c-66cf5f2cd9ab tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.482875 Oct 04 08:36:09.927295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 62/123] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:09 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:10.399669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-f39c4383-a415-496c-813d-f616148e4b79 req-e3a534a4-8330-47f2-9859-028f692686a7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54819895-e219-4341-8f1e-5543b7357315", "tag": "159277a9-e8de-47c4-813b-8de31279c18f"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:10.423838 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:10.425518 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:10.426878 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:10.436851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:10.440448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:10.441371 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:10.442931 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f39c4383-a415-496c-813d-f616148e4b79 req-e3a534a4-8330-47f2-9859-028f692686a7 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:10.443433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f39c4383-a415-496c-813d-f616148e4b79 req-e3a534a4-8330-47f2-9859-028f692686a7 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:10.443868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f39c4383-a415-496c-813d-f616148e4b79 req-e3a534a4-8330-47f2-9859-028f692686a7 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:10.444539 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:10.445219 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:10.447091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:10.475575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-f39c4383-a415-496c-813d-f616148e4b79 req-e3a534a4-8330-47f2-9859-028f692686a7 service nova] Creating event network-changed:159277a9-e8de-47c4-813b-8de31279c18f for instance 54819895-e219-4341-8f1e-5543b7357315 on hv-cinder-86020 Oct 04 08:36:10.486455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-f39c4383-a415-496c-813d-f616148e4b79 req-e3a534a4-8330-47f2-9859-028f692686a7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.097185 Oct 04 08:36:10.489337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 62/124] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:36:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:36:10.952322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:10.953024 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:10.971739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:10.972233 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:10.972667 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:10.974159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:10.974707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:10.975078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:11.535792 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:11.550868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c76f26d5-29f4-48b5-a81d-00b65ae31204 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1366 microversion: 2.1 time: 1.130716 Oct 04 08:36:11.562590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 63/125] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:10 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1366 bytes in 1136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:11.753054 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:11.774929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-67f4f75a-669e-4a8b-8770-bcb8b6a2547e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.824211 Oct 04 08:36:11.779744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 63/126] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:10 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 830 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:12.256087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-929961e9-c2f9-4d6e-9e34-29db35ec51da req-c8163a61-78bd-4f51-9974-e1840355c490 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9fd4027-9ae5-4ab2-afc6-c7eb723204cf", "name": "network-vif-plugged", "status": "completed", "tag": "364e2511-1153-48e7-8595-032dbdba4297"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:12.272802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-929961e9-c2f9-4d6e-9e34-29db35ec51da req-c8163a61-78bd-4f51-9974-e1840355c490 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:12.273703 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-929961e9-c2f9-4d6e-9e34-29db35ec51da req-c8163a61-78bd-4f51-9974-e1840355c490 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:12.276135 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-929961e9-c2f9-4d6e-9e34-29db35ec51da req-c8163a61-78bd-4f51-9974-e1840355c490 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:12.315655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-929961e9-c2f9-4d6e-9e34-29db35ec51da req-c8163a61-78bd-4f51-9974-e1840355c490 service nova] Creating event network-vif-plugged:364e2511-1153-48e7-8595-032dbdba4297 for instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf on hv-cinder-86020 Oct 04 08:36:12.327858 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-929961e9-c2f9-4d6e-9e34-29db35ec51da req-c8163a61-78bd-4f51-9974-e1840355c490 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075794 Oct 04 08:36:12.329212 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 64/127] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:36:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:36:12.576549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:12.577242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:12.618472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:12.620951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:12.621326 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:12.622035 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:12.622475 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:12.627831 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:12.802416 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:12.803245 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:12.810074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:12.810360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:12.811440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:12.812085 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:12.812995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:12.813678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:12.814291 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:12.819297 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0e0ee685-3750-45d1-b4f1-f7cb738f5308 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1556 microversion: 2.1 time: 0.246566 Oct 04 08:36:12.834583 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 64/128] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:12 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1556 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:12.845019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:12.845681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:12.853100 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:12.853641 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:12.854088 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:12.854820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:12.855461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:12.855890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:12.973647 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-67389f2c-d5b6-4171-b81f-67cafffd0126 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.132392 Oct 04 08:36:12.974459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 65/129] 10.222.0.16 () {58 vars in 1275 bytes} [Tue Oct 4 08:36:12 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-interface => generated 284 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:36:13.015151 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:13.024289 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2d34b588-ddb3-40c4-972a-4a6d5885baa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.226693 Oct 04 08:36:13.024881 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 65/130] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:12 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:14.046735 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:14.047749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:14.054451 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:14.055131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:14.055766 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:14.056456 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:14.057126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:14.057697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:14.208479 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:14.216634 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-64f79302-7d5f-4f59-b69b-523ceb8f01ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.173643 Oct 04 08:36:14.218489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 66/131] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:14 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:14.429341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-d1cd6837-45c7-4881-8516-f15f92ea6513 req-9ae41a75-08cc-4072-b8f7-d0657869a36d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8351183-2123-48d0-9f68-c06ab2829d71", "tag": "4b6d91d8-8d57-437a-8e26-84b89843a14e"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:14.443169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-d1cd6837-45c7-4881-8516-f15f92ea6513 req-9ae41a75-08cc-4072-b8f7-d0657869a36d service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:14.443642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-d1cd6837-45c7-4881-8516-f15f92ea6513 req-9ae41a75-08cc-4072-b8f7-d0657869a36d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:14.444117 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-d1cd6837-45c7-4881-8516-f15f92ea6513 req-9ae41a75-08cc-4072-b8f7-d0657869a36d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:14.459422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-d1cd6837-45c7-4881-8516-f15f92ea6513 req-9ae41a75-08cc-4072-b8f7-d0657869a36d service nova] Creating event network-changed:4b6d91d8-8d57-437a-8e26-84b89843a14e for instance a8351183-2123-48d0-9f68-c06ab2829d71 on hv-cinder-86020 Oct 04 08:36:14.464401 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-d1cd6837-45c7-4881-8516-f15f92ea6513 req-9ae41a75-08cc-4072-b8f7-d0657869a36d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041096 Oct 04 08:36:14.465002 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 66/132] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:36:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:36:15.240942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:15.241724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:15.253620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:15.254342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:15.259326 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:15.260064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:15.260525 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:15.260951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:15.514223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:15.522222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b81910f2-4492-47cb-bf43-426898338fdd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.286754 Oct 04 08:36:15.522892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 67/133] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:15 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:16.550225 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:16.552331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:16.558674 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:16.559375 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:16.559809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:16.560661 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:16.561264 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:16.561836 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:16.746219 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:16.755852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c9f49975-a4dc-42e5-a020-990f069941cf tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.214352 Oct 04 08:36:16.756442 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 67/134] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:16 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:17.781367 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:17.782095 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:17.788308 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:17.788789 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:17.789219 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:17.789905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:17.790351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:17.790797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:17.926887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:17.934594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c6803e6d-d7cb-48f7-936d-ffa14c5b1acd tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.157015 Oct 04 08:36:17.935177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 68/135] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:17 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:18.958152 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:18.958798 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:18.961266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=68 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:18.966633 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:18.967127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:18.967558 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:18.968269 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:18.968698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:18.969118 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:19.103238 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:19.110934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9de22b05-a347-4699-9b95-0b3994ee5030 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.157562 Oct 04 08:36:19.111516 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 68/136] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:18 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:20.134297 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:20.134935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:20.145500 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:20.145500 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:20.145500 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:20.145500 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:20.145962 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:20.145962 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:20.298192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:20.306254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-daec01c0-777c-4c33-937c-f08fde19294b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.175782 Oct 04 08:36:20.306881 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 69/137] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:20 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:21.329006 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:21.329552 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:21.337117 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:21.337619 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:21.338043 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:21.338780 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:21.339234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:21.339656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:21.355711 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:21.481775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:21.489971 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a10d80e8-95e6-47f3-9178-44034be0ba99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.164943 Oct 04 08:36:21.490571 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 69/138] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:21 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:22.513288 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:22.513953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:22.521247 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:22.521729 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:22.522163 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:22.522905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:22.523353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:22.523772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:22.528299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:22.661860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:22.669783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-47d1b2fb-3d9a-4ddb-a1a1-79920f1bf6f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.160560 Oct 04 08:36:22.670349 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 70/139] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:22 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:23.694445 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:23.695316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:23.701639 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:23.702126 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:23.702576 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:23.703278 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:23.703737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:23.704150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:23.839263 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:23.846888 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0006ac09-683c-4828-9c9c-6fd0a0e89bc8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.157920 Oct 04 08:36:23.848250 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 70/140] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:23 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:24.869882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:24.870504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:24.877159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:24.877642 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:24.878073 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:24.878864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:24.879895 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:24.880314 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:25.013347 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:25.023478 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4733ce27-546a-498b-a66b-403081247895 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.157314 Oct 04 08:36:25.029483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 71/141] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:24 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:26.046334 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:26.046929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:26.054209 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:26.054725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:26.055150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:26.055826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:26.056269 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:26.056680 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:26.186791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:26.195415 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ab8bebbd-5c87-419c-9dbe-f6c7e3f01cb0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1322 microversion: 2.1 time: 0.152941 Oct 04 08:36:26.195987 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 71/142] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:26 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1322 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:27.218727 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:27.219557 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:27.228922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:27.229679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:27.230774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:27.230980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:27.231432 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:27.231854 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:27.460134 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:27.470534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d52e4bd3-33d4-4d49-92e7-ac6873a92b43 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1515 microversion: 2.1 time: 0.255627 Oct 04 08:36:27.471286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 72/143] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:27 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1515 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:27.495948 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:27.498124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:27.505289 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:27.505776 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:27.506404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:27.507153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:27.507611 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:27.509053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:27.535554 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'flavor' on Instance uuid b9fd4027-9ae5-4ab2-afc6-c7eb723204cf {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:36:27.679428 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1bb22003-2436-4312-ba09-0c511d298a78 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf/action" status: 200 len: 32 microversion: 2.1 time: 0.187409 Oct 04 08:36:27.680047 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 72/144] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:36:27 2022] POST /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf/action => generated 32 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 04 08:36:27.696972 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:27.697590 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:27.712128 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:27.713120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:27.713627 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:27.714593 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:27.715256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:27.715682 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:27.850906 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:27.858815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-837e9179-5f80-409d-a701-86568469ed67 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.171457 Oct 04 08:36:27.859385 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 73/145] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:27 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:28.053187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:28.053808 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:28.060664 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:28.061250 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:28.061680 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:28.062372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:28.062851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:28.063273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:28.089713 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:36:28.185153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b8fee740-85ae-40e1-b79b-4bef83106938 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 204 len: 0 microversion: 2.1 time: 0.137175 Oct 04 08:36:28.185740 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 73/146] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:36:28 2022] DELETE /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:36:28.194356 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:28.194989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:28.200931 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:28.201396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:28.201821 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:28.202500 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:28.204503 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:28.204660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:28.346617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:28.355641 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b43ff456-da8f-474a-bb8a-f34e1ae12dad tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1553 microversion: 2.1 time: 0.165574 Oct 04 08:36:28.356222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 74/147] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:28 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1553 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:28.886170 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:28.886889 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:28.893770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:28.895122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:28.895568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:28.896277 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:28.896725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:28.897145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:29.037684 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:29.046121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-22e3d1db-ca01-44ae-bbd6-97b638f051b7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.164123 Oct 04 08:36:29.046828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 74/148] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:28 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:29.379711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:29.380571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:29.389215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:29.389683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:29.390108 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:29.390854 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:29.391297 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:29.391708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:29.560057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:29.569199 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7e87431e-0d3c-4105-917a-711de0f8a623 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1553 microversion: 2.1 time: 0.193502 Oct 04 08:36:29.569771 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 75/149] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:29 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1553 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:30.069489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:30.070002 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:30.076405 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:30.076873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:30.077294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:30.077968 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:30.078409 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:30.078856 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:30.214973 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:30.222656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-15853dc8-aa73-4dee-8841-42d6d89e2a0e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.157161 Oct 04 08:36:30.223235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 75/150] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:30 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:30.592780 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:30.593377 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:30.599389 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:30.599862 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:30.600280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:30.600955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:30.601383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:30.601794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:30.731418 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:30.739817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-801d3483-0f32-4c15-a70d-952f550bf4d7 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1553 microversion: 2.1 time: 0.150760 Oct 04 08:36:30.740319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 76/151] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:30 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1553 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:31.245729 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:31.246362 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:31.253046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:31.254259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:31.254524 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:31.255328 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:31.255768 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:31.256180 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:31.386699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:31.394350 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-094f56ee-9a33-45d6-900a-3d2b59143e7c tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.152645 Oct 04 08:36:31.394903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 76/152] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:31 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:31.763295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:31.763915 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:31.772031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:31.772874 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:31.773826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:31.774873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:31.775330 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:31.776232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:31.987371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:31.996981 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8f035ae2-f018-424b-bb3a-a5b7a41cc050 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1553 microversion: 2.1 time: 0.237376 Oct 04 08:36:31.997552 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 77/153] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:31 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1553 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:32.418261 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:32.420357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:32.430741 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:32.431564 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:32.432090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:32.433226 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:32.433940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:32.435292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:32.716170 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:32.725732 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6eb90d21-38c7-4ed3-b44d-b3432fa160ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.311743 Oct 04 08:36:32.726312 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 77/154] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:32 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:33.024117 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:33.025190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:33.033216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:33.033812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:33.034315 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:33.035398 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:33.036070 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:33.037104 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:33.410109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:33.424080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f458ab0d-265d-4c08-82dd-cf1077be7c11 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1553 microversion: 2.1 time: 0.406025 Oct 04 08:36:33.424690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 78/155] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:33 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1553 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:33.753231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:33.754382 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:33.766960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:33.767539 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:33.769785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:33.774174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:33.774930 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:33.775966 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:34.107071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:34.114904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8559cb55-fbde-4301-b73b-e92619f8dee6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.366189 Oct 04 08:36:34.115476 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 78/156] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:33 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:34.324120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-46878d50-1be6-4224-b4eb-b1766c7dc70c req-d0406824-ad4d-4020-b381-fa3e7be38f9d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7bf8ac1b-807c-4209-aab2-2d6aff4038f0", "name": "network-vif-deleted", "tag": "52f10973-3a78-447b-a7be-d1bc8b3c7e32"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:36:34.336259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-46878d50-1be6-4224-b4eb-b1766c7dc70c req-d0406824-ad4d-4020-b381-fa3e7be38f9d service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:34.336738 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-46878d50-1be6-4224-b4eb-b1766c7dc70c req-d0406824-ad4d-4020-b381-fa3e7be38f9d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:34.337168 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-46878d50-1be6-4224-b4eb-b1766c7dc70c req-d0406824-ad4d-4020-b381-fa3e7be38f9d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:34.350859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-46878d50-1be6-4224-b4eb-b1766c7dc70c req-d0406824-ad4d-4020-b381-fa3e7be38f9d service nova] Creating event network-vif-deleted:52f10973-3a78-447b-a7be-d1bc8b3c7e32 for instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 on hv-cinder-86020 Oct 04 08:36:34.356728 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-46878d50-1be6-4224-b4eb-b1766c7dc70c req-d0406824-ad4d-4020-b381-fa3e7be38f9d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036432 Oct 04 08:36:34.356861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 79/157] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:36:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:36:34.450595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:34.451216 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:34.457389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:34.457876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:34.458344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:34.459089 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:34.459546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:34.459966 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:34.527998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:34.537756 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-02b8b86c-71ab-4f11-930b-c0b81020a9ce tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1511 microversion: 2.1 time: 0.091000 Oct 04 08:36:34.538412 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 79/158] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:34 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1511 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:35.139277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:35.139970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:35.141965 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=48 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:35.147985 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:35.148463 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:35.148888 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:35.149579 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:35.150021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:35.150434 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:35.289701 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:35.297613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-84dd2fae-608a-4a5b-a567-309070190d00 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.162820 Oct 04 08:36:35.298188 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 80/159] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:35 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:35.564138 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:35.564678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:35.571182 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:35.571664 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:35.572100 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:35.572790 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:35.573243 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:35.573661 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:35.634427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:35.642840 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7fed60af-e771-4f6d-b546-fff97689fb9c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1345 microversion: 2.1 time: 0.082628 Oct 04 08:36:35.643415 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 80/160] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:35 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1345 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:36.322279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:36.322916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:36.328850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:36.329318 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:36.329774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:36.330443 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:36.330938 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:36.331351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:36.462615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:36.470423 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-54a96fdb-bb22-4302-a365-46d8f6d3f130 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.151983 Oct 04 08:36:36.471033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 81/161] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:36 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:36.666732 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:36.667266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:36.673682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:36.674164 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:36.674639 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:36.676144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:36.676850 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:36.677295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:36.741259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:36.749750 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-202504ab-7eed-4889-ae10-570885295aea tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1345 microversion: 2.1 time: 0.087065 Oct 04 08:36:36.750316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 81/162] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:36 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1345 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:37.494783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:37.495395 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:37.502031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:37.502883 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:37.503930 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:37.504763 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:37.505211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:37.505625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:37.639494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:37.648007 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-bdb86c98-c8d5-453f-9a83-f6ef82fa463b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.157379 Oct 04 08:36:37.648564 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 82/163] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:37 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:37.773321 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:37.773902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:37.781130 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:37.781608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:37.782038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:37.782746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:37.783196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:37.783651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:37.798395 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:37.851801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:37.860024 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a9dc1fc2-f3b2-4b3f-af0b-37d91972a28d tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1345 microversion: 2.1 time: 0.091503 Oct 04 08:36:37.860588 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 82/164] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:37 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1345 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:38.671464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:38.672039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:38.678889 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:38.679375 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:38.679807 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:38.680499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:38.680938 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:38.681352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:38.821632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:38.830499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7346ce6f-d19d-4efd-b8b8-f59b7c24aba5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.162740 Oct 04 08:36:38.831051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 83/165] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:38 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:38.883422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:38.884012 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:38.890095 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:38.890596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:38.891029 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:38.891707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:38.892150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:38.892565 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:38.954288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:38.963279 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3d8333f5-3bcd-462d-be4f-29f2d694a006 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1345 microversion: 2.1 time: 0.083601 Oct 04 08:36:38.963865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 83/166] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:38 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1345 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:39.853888 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:39.854445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:39.860338 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:39.860806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:39.861231 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:39.861896 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:39.862335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:39.862876 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:39.986498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:39.987132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:39.990299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:39.994699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:39.995175 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:39.995606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:39.996290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:39.996766 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:39.997182 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:39.997740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cb23e4a0-074e-46ba-8fee-aa83aa47185e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.147761 Oct 04 08:36:39.998370 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 84/167] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:39 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:40.058940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:40.067660 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4ca1e0c1-3796-4965-be04-083309d06145 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1345 microversion: 2.1 time: 0.085134 Oct 04 08:36:40.068223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 84/168] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:39 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1345 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:41.021594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:41.022227 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:41.028198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:41.028672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:41.029100 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:41.029775 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:41.030218 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:41.030662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:41.091410 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:41.092133 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:41.099187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:41.099746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:41.100261 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:41.101030 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:41.101550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:41.102038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:41.142465 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:41.143002 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:41.143441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:41.143895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lazy-loading 'fault' on Instance uuid 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:36:41.186441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:41.195520 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-dbe20ba6-f73c-4f50-95a8-6c365f1e7e4f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.178211 Oct 04 08:36:41.196086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 85/169] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:41 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:41.204835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd002a844a71649d884b4ca870eb9d387', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f34eba992024f6881d1292c9f7cfd48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:41.214598 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-353ce449-d261-447a-92ff-8775e5750b33 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 200 len: 1350 microversion: 2.1 time: 0.127713 Oct 04 08:36:41.215185 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 85/170] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:41 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 1350 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:42.226574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:42.227415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:42.238761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:42.239481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:42.240878 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:42.242296 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:42.242882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:42.243837 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:42.247554 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:42.248412 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:42.261753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:42.263310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:42.264665 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:42.265818 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:42.266453 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:42.266961 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:42.287415 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] HTTP exception thrown: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Request method failure captured: Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 HTTP/1.1 Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BHc9C2loMJrmI5IDkPdC3oblxDQBHttR-DM8TSJTikOfI2F8NuMms-jHuEuXXzFyffsV2dyn8_S6zjZAUgdnd7z3b-KGqZ2-sOZRyacmmf-6e0AzNDQt8H0Li5-oNtMyAqkHtaKgGXNTkWXfjsDrqyH7P6bv-V0I68OYIfEW-UtTQ Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:36:42.294366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3f34eba992024f6881d1292c9f7cfd48", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: d002a844a71649d884b4ca870eb9d387 Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.296163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': '7bf8ac1b-807c-4209-aab2-2d6aff4038f0'} Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:36:42.297192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.298535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.302444 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Returning 404 to user: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:36:42.303777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-65a35937-1b64-448f-9485-0af1a8b6f725 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 404 len: 111 microversion: 2.1 time: 0.062747 Oct 04 08:36:42.304734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 86/171] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:42 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:36:42.493085 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:42.507278 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:42.511544 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-98c878da-babb-4827-a6b2-13808ea5475e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.290596 Oct 04 08:36:42.512464 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:42.514001 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 86/172] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:42 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:42.524617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:42.525685 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:42.526403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:42.528090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:42.528998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:42.529669 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:42.555348 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] HTTP exception thrown: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Request method failure captured: Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0/os-volume_attachments HTTP/1.1 Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BHc9C2loMJrmI5IDkPdC3oblxDQBHttR-DM8TSJTikOfI2F8NuMms-jHuEuXXzFyffsV2dyn8_S6zjZAUgdnd7z3b-KGqZ2-sOZRyacmmf-6e0AzNDQt8H0Li5-oNtMyAqkHtaKgGXNTkWXfjsDrqyH7P6bv-V0I68OYIfEW-UtTQ Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:36:42.558163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3f34eba992024f6881d1292c9f7cfd48", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: d002a844a71649d884b4ca870eb9d387 Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.559528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'server_id': '7bf8ac1b-807c-4209-aab2-2d6aff4038f0'} Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:36:42.560186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:42.561249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:42.566020 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Returning 404 to user: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:36:42.566020 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-54a07f90-495c-45a8-a8e9-1f63f3de836b tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.062039 Oct 04 08:36:42.566020 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 87/173] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:36:42 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0/os-volume_attachments => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:36:43.541175 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:43.541803 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:43.548224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:43.548919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:43.549636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:43.551222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:43.551983 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:43.552562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:43.574211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:43.824162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:43.841116 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-bafb7908-1852-47e5-9f0e-ada379ac564b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.303753 Oct 04 08:36:43.841835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 87/174] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:43 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:43.973559 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:43.975266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:43.988139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:43.989491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:43.990500 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:43.991869 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:43.992904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:43.993940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:44.016045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] HTTP exception thrown: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Request method failure captured: Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: DELETE /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 HTTP/1.1 Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BHc9C2loMJrmI5IDkPdC3oblxDQBHttR-DM8TSJTikOfI2F8NuMms-jHuEuXXzFyffsV2dyn8_S6zjZAUgdnd7z3b-KGqZ2-sOZRyacmmf-6e0AzNDQt8H0Li5-oNtMyAqkHtaKgGXNTkWXfjsDrqyH7P6bv-V0I68OYIfEW-UtTQ Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:36:44.018273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3f34eba992024f6881d1292c9f7cfd48", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: d002a844a71649d884b4ca870eb9d387 Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.021054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': '7bf8ac1b-807c-4209-aab2-2d6aff4038f0'} Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:36:44.022267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.027031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.031613 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Returning 404 to user: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:36:44.031613 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8d956e86-89b7-429a-8f6d-7829b992b54c tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 404 len: 111 microversion: 2.1 time: 0.055222 Oct 04 08:36:44.031613 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 88/175] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:36:43 2022] DELETE /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:36:44.034930 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:44.035758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] [instance: 7bf8ac1b-807c-4209-aab2-2d6aff4038f0] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:44.044236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:44.044833 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:44.045511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:44.046520 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:44.047496 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:44.047957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:44.072430 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] HTTP exception thrown: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Request method failure captured: Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 HTTP/1.1 Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BHc9C2loMJrmI5IDkPdC3oblxDQBHttR-DM8TSJTikOfI2F8NuMms-jHuEuXXzFyffsV2dyn8_S6zjZAUgdnd7z3b-KGqZ2-sOZRyacmmf-6e0AzNDQt8H0Li5-oNtMyAqkHtaKgGXNTkWXfjsDrqyH7P6bv-V0I68OYIfEW-UtTQ Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:36:44.078223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3f34eba992024f6881d1292c9f7cfd48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3f34eba992024f6881d1292c9f7cfd48", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 3f34eba992024f6881d1292c9f7cfd48 Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-DeleteServersTestJSON-1506415323 Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: d002a844a71649d884b4ca870eb9d387 Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-DeleteServersTestJSON-1506415323-project Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.079802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '7bf8ac1b-807c-4209-aab2-2d6aff4038f0'} Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:36:44.080951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. Oct 04 08:36:44.082802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:36:44.084108 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] Returning 404 to user: Instance 7bf8ac1b-807c-4209-aab2-2d6aff4038f0 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:36:44.084108 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1c6d67a2-479b-4f20-b0ef-d4c300e618a5 tempest-DeleteServersTestJSON-1506415323 tempest-DeleteServersTestJSON-1506415323-project] 10.222.0.16 "GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0" status: 404 len: 111 microversion: 2.1 time: 0.052807 Oct 04 08:36:44.087612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 88/176] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:44 2022] GET /compute/v2.1/servers/7bf8ac1b-807c-4209-aab2-2d6aff4038f0 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:36:44.883056 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:44.886160 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:44.896859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:44.898876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:44.900194 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:44.901392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:44.902610 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:44.903392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:45.122043 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:45.157290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c45840d8-4a30-4ec0-bb0b-28ae73a3c2af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.279535 Oct 04 08:36:45.158759 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 89/177] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:44 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:46.213177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:46.218502 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:46.248324 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:46.252516 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:46.253628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:46.255097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:46.260318 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:46.262916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:46.581736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:46.590924 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-74e50c83-5b32-4ad9-9370-748b7f379c5b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.387879 Oct 04 08:36:46.593476 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 89/178] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:46 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:47.636814 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:47.638111 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:47.655184 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:47.655937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:47.656562 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:47.657450 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:47.658218 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:47.659081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:48.048053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:48.069715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-338252ed-ee39-45ff-bf25-9d6656b527ac tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.437049 Oct 04 08:36:48.086722 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 90/179] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:47 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:49.129158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:49.130345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:49.149872 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:49.150918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:49.152912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:49.156216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:49.157625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:49.158837 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:49.402152 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:49.419287 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5ae3d9a0-027e-4dd8-91d3-e140023765af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.310226 Oct 04 08:36:49.420727 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 90/180] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:49 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:50.444976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:50.446079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:50.459461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:50.467482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:50.472125 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:50.474437 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:50.479306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:50.482581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:50.483477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:51.229455 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:51.264353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-24ddb517-2ede-4059-99b7-b9a452474da4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.824319 Oct 04 08:36:51.270182 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 91/181] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:50 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:52.308346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:52.319094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:52.349982 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:52.351409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:52.352113 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:52.353358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:52.353960 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:52.354476 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:53.075250 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:53.104106 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-615b9044-efbb-4601-86f3-b7b40d461c8d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.811572 Oct 04 08:36:53.118025 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 91/182] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:52 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 815 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:54.137781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:54.139276 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:54.196009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:54.201074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:54.203246 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:54.226384 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:54.230640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:54.237867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:54.877629 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:54.898716 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-12dd1dd8-f5e0-4c2f-9375-b992213f1f45 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.768043 Oct 04 08:36:54.920679 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 92/183] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:54 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:55.927312 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:55.928088 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:55.935521 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:55.936143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:55.936936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:55.938149 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:55.938529 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:55.939019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:56.180752 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:56.197222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a8e602b2-7e74-438e-bbe9-f342ea321d36 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.274658 Oct 04 08:36:56.203068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 92/184] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:55 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:36:57.229006 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:57.230481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:57.242726 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:57.243513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:57.244112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:57.245384 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:57.246013 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:57.246615 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:57.617878 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:57.629817 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b8602956-4cb5-4bcc-a288-629e4d4524f0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.405566 Oct 04 08:36:57.633038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 93/185] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:57 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:36:58.663675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:36:58.664740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:36:58.701044 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:58.701583 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:58.702093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:58.702862 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:36:58.703420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:36:58.703861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:36:59.264632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:36:59.283154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-98f5aa6a-367d-47ea-9287-1b64a5128c7e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.625937 Oct 04 08:36:59.286976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 93/186] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:36:58 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:00.314629 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:00.316098 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:00.325848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:00.326935 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:00.327895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:00.340375 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:00.347523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:00.349934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:01.487139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:01.501557 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ed09d76f-6659-4616-97b3-2dabf97369f9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 1.192692 Oct 04 08:37:01.503840 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 94/187] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:00 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 1196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:02.533604 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:02.540719 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:02.554829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:02.557681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:02.558683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:02.559886 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:02.561791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:02.562816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:03.074639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:03.089930 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ab32e55b-a56c-442d-9d3a-662787d92fc4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.561464 Oct 04 08:37:03.099173 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 94/188] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:02 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:04.130167 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:04.131480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:04.139260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:04.139935 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:04.147603 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:04.148646 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:04.156939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:04.157775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:05.027064 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:05.046754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-102defaa-9ee1-453a-abfa-ab4720ec130e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.920651 Oct 04 08:37:05.062959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 95/189] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:04 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 938 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:06.077809 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:06.089876 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:06.115488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=30 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:06.127089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:06.130165 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:06.134534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:06.136774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:06.137983 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:06.148010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:06.264139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:06.264933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:06.280494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:06.280995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:06.284775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:06.285769 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:06.286676 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:06.289720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:06.545144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-db7b8f78-baea-440f-b8f0-944dcf5fad21 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/ips" status: 200 len: 153 microversion: 2.20 time: 0.286065 Oct 04 08:37:06.553987 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 96/190] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:37:06 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/ips => generated 153 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:37:06.940846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:06.956065 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9af10ca6-5ca2-4bbe-981f-07049f53c879 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1521 microversion: 2.1 time: 0.883135 Oct 04 08:37:06.956673 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 95/191] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:06 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1521 bytes in 885 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:07.983064 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:07.990873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:08.004778 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:08.008391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:08.011071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:08.012320 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:08.013744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:08.014432 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:08.019910 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=76 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:08.466589 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:08.479512 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-02c7137e-a2fd-4e5d-8a3f-3b4c5bd195a6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1501 microversion: 2.1 time: 0.500707 Oct 04 08:37:08.485565 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 97/192] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:07 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1501 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:08.516714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:08.517393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:08.529749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:08.530629 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:08.532388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:08.534482 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:08.538189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:08.539826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:09.117086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:09.126590 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0d58c9a1-e5f7-4af2-9531-0edec314888b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1516 microversion: 2.1 time: 0.613907 Oct 04 08:37:09.127174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 96/193] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:08 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1516 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:09.229942 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:09.231468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:09.246865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:09.247582 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:09.248246 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:09.249449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:09.252538 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:09.253715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:09.340985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lazy-loading 'flavor' on Instance uuid 54819895-e219-4341-8f1e-5543b7357315 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:37:09.718431 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:09.719060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:09.756642 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:09.757951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:09.758644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:09.761634 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:09.773178 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:09.781330 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:09.928475 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c0fb0240-83ba-4572-bba7-4bb78d92f11a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/ips" status: 200 len: 149 microversion: 2.1 time: 0.207150 Oct 04 08:37:09.929697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 97/194] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:37:09 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/ips => generated 149 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:37:10.174377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c35243d8-0f21-40e5-9424-caece4c63967 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "POST /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/action" status: 202 len: 0 microversion: 2.20 time: 0.947911 Oct 04 08:37:10.183011 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 98/195] 10.222.0.16 () {62 vars in 1319 bytes} [Tue Oct 4 08:37:09 2022] POST /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/action => generated 0 bytes in 958 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:10.189399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:10.190224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:10.200942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:10.207426 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:10.208219 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:10.209267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:10.211028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:10.218519 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:11.156100 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:11.179102 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:11.184292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:11.185717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2c4b9e0b-417d-4fe0-b339-fc59bd57ef9e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 1.001096 Oct 04 08:37:11.190892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 98/196] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:10 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 1006 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:11.313595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c132e52a-daa1-4bb9-8a3d-d366d98b92b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'create', calling method: .version_select at 0x7f30a0cd30d0>, body: {"keypair": {"name": "tempest-keypair-2045943656", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAz6HeJP2yCMl1IyFlDbChvyOiCQr1qMOAWwFSGZw7+cbzIy/uW2sFemkFr+A543hWbWHQW9mjrDtI6u9x775n+p3IsZ6JfH3UuramO+07oegQ5hRENefgVDDSdw7IEKMg=="}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:11.357377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-c132e52a-daa1-4bb9-8a3d-d366d98b92b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for project 5d21789971964770bdf417547cd19840. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:11.370924 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-c132e52a-daa1-4bb9-8a3d-d366d98b92b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for user 78634b34107d49cdbf0126014bf8f6f5 and project 5d21789971964770bdf417547cd19840. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:11.429360 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c132e52a-daa1-4bb9-8a3d-d366d98b92b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.900001 Oct 04 08:37:11.431494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 99/197] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:37:10 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 903 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:11.499049 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:11.513294 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:11.524539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:11.525417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:11.526961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:11.540546 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:11.541193 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:11.541663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:11.600859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'flavor' on Instance uuid f597e06f-510c-46fe-92de-ce4a8dc53841 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:37:11.856494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b246e0bf-505c-45ea-aa5e-3dd4ee8e4fe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841/action" status: 200 len: 32 microversion: 2.1 time: 0.379510 Oct 04 08:37:11.861175 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 99/198] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:37:11 2022] POST /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841/action => generated 32 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:37:11.885218 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:11.889339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:11.907461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:11.911122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:11.911863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:11.914982 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:11.917514 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:11.918289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:12.312665 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-94f733e3-c7e7-46d9-bd07-f7095d639cc7 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Action: 'create', calling method: .version_select at 0x7f30a0a6caf0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-2095819453", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC6jaWFCnFU6AypS/tDDU0e4U+p7y1La2M0WaO5jZRCMnqhJFoVufdk8N8iwbZQOrscKs/esDIKnLWdqZiJtTjBjVu+ZbKTje1gRZO7w8x+58pfWAI8FuScuWu6rSsR03g=="}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:12.355311 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-94f733e3-c7e7-46d9-bd07-f7095d639cc7 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Getting quotas for project 2f94e3ae140b43739febe1111e18604b. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:12.362999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-94f733e3-c7e7-46d9-bd07-f7095d639cc7 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Getting quotas for user ab4b3c2b3f4b4978b0eb472f9c2736b1 and project 2f94e3ae140b43739febe1111e18604b. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:12.403466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-94f733e3-c7e7-46d9-bd07-f7095d639cc7 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.637879 Oct 04 08:37:12.414729 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 100/199] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:37:11 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:12.427683 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7fc3975c-c233-43c0-aa63-cdcf158a4760"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:12.432129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:12.476200 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:12.477536 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:12.481726 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:12.484653 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:12.487489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:12.488859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:12.622854 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:12.647961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-02971fea-aeb9-4ca4-bc28-89d0acfa9654 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.780942 Oct 04 08:37:12.653769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 100/200] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:11 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:12.679348 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:12.681417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:12.706679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:12.708480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:12.709467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:12.728724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:12.732153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:12.737178 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:13.061046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3582ceb0-2019-412c-9f01-76753599d494 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.644305 Oct 04 08:37:13.072452 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 101/201] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:37:12 2022] POST /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments => generated 194 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:13.679596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:13.680060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:13.694722 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:13.695219 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:13.695653 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:13.696377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:13.696830 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:13.697254 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:14.098215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:14.103747 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:14.132824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:14.138743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:14.144015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9910e941-daab-40e7-bf74-ef3733cb3eae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 1.486612 Oct 04 08:37:14.146797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 101/202] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:12 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 1493 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:14.804480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:14.829720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bf733f87-1157-4356-b450-1865dc27f190 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 1.156126 Oct 04 08:37:14.830470 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 102/203] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:13 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 1158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:15.177786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:15.179108 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:15.199816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:15.200327 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:15.203989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:15.204755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:15.205222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:15.206252 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:15.344992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-576500020", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "networks": [{"uuid": "1ee6d49b-e78d-48c2-9f17-a27346795ae3"}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:15.512725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] validate_networks() for [('1ee6d49b-e78d-48c2-9f17-a27346795ae3', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:37:15.791563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:15.819293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:15.828967 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:15.833231 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2cb3df89-02bd-4d39-895f-33bf7b93597a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.661929 Oct 04 08:37:15.837174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 102/204] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:15 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:15.865976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:15.866782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:15.891593 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:15.892358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:15.894945 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:15.896488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:15.897156 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:15.897812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:16.493818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:16.507639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-87c40260-61bd-4218-b1c7-f6b6e2841379 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.645965 Oct 04 08:37:16.516591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 103/205] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:15 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:16.594105 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:37:16.595455 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:37:16.595777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:37:16.596351 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:37:16.596791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:37:16.639017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:16.640230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:16.692614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.052s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:16.693959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:16.698946 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:16.708997 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:16.778842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Getting quotas for project f758e7615232486a93a915760633b702. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:16.790040 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Getting quotas for user 76b2078f1f64440e9d8faca3bcf98d52 and project f758e7615232486a93a915760633b702. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:16.810137 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:37:16.820075 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:37:16.867914 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:16.869067 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:16.878165 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:16.881458 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:16.883258 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:16.893342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:16.894501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:16.895420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:17.090271 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-41dc7ba7-c887-4272-a055-7c54012e5da0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.749568 Oct 04 08:37:17.104740 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 103/206] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:37:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 1765 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 04 08:37:17.127070 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-38e90e88-d8f5-4255-9bf0-9d5474e7f85a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:17.143810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-38e90e88-d8f5-4255-9bf0-9d5474e7f85a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:17.228664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:17.250422 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:17.254178 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:17.262527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-bac96b24-7f2f-4322-b17c-71c2a0489902 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.396188 Oct 04 08:37:17.289733 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 104/207] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:16 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:37:17.313317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-38e90e88-d8f5-4255-9bf0-9d5474e7f85a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:17.331548 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-38e90e88-d8f5-4255-9bf0-9d5474e7f85a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1213 microversion: 2.1 time: 0.214117 Oct 04 08:37:17.336127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 104/208] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:17 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1213 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:17.493841 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-327301408", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd08aa17-8a80-4dac-a656-3195d8433ec8"}], "security_groups": [{"name": "tempest-securitygroup--1235588277"}], "key_name": "tempest-keypair-2045943656", "user_data": "}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:17.550164 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:17.552730 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:17.566262 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:17.570731 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:17.574746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:17.577593 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:17.579493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:17.589699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:18.118260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] validate_networks() for [('fd08aa17-8a80-4dac-a656-3195d8433ec8', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:37:18.159670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:18.169929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b3b890c6-42a8-4353-be4b-ba2b56bd91d6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.630545 Oct 04 08:37:18.171139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 105/209] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:17 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:18.302873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:18.303557 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:18.316413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:18.319556 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:18.320279 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:18.321053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:18.321523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:18.321956 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:18.850430 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:18.875981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:18.878059 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:18.885038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-75c16e58-6314-48e2-8991-788eac399dc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.587193 Oct 04 08:37:18.912456 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 106/210] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:18 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:18.919929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:18.922738 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:18.944498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:18.946894 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:18.947584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:18.948508 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:18.949117 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:18.950356 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:19.079535 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:37:19.080097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:37:19.081585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:37:19.082033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:37:19.082443 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:37:19.099658 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:19.100131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:19.101059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:19.102006 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:19.102459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:19.102910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:19.118534 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:19.123234 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for project 5d21789971964770bdf417547cd19840. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:19.128828 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for user 78634b34107d49cdbf0126014bf8f6f5 and project 5d21789971964770bdf417547cd19840. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:19.139820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-eb04c14c-a420-442a-9e0e-c23ab3778d47 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1265 microversion: 2.1 time: 0.216687 Oct 04 08:37:19.140504 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 107/211] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:18 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1265 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:19.178625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:37:19.181671 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:37:19.200634 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:19.201256 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:19.214938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:19.215481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:19.216105 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:19.217037 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:19.217737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:19.218323 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:19.250289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7b27cc21-321b-42b7-9da8-d16488d65dd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.762062 Oct 04 08:37:19.250964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 105/212] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:37:17 2022] POST /compute/v2.1/servers => generated 402 bytes in 1764 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:37:19.268391 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3f3ebd04-fa68-4351-9773-40e07442804b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:19.269303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-3f3ebd04-fa68-4351-9773-40e07442804b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:19.390504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-3f3ebd04-fa68-4351-9773-40e07442804b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:19.405862 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3f3ebd04-fa68-4351-9773-40e07442804b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1237 microversion: 2.1 time: 0.146153 Oct 04 08:37:19.411336 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 106/213] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:19 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1237 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:19.558262 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:19.568666 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d83b74be-759a-4d04-95d7-2d276383f7b2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.372188 Oct 04 08:37:19.585631 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 108/214] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:19 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:19.915091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:19.915715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:19.922135 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:19.922719 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:19.923427 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:19.924783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:19.925300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:19.925740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:20.105351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:20.119634 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:20.121795 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:20.123657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-909d0c5b-8e45-4c62-8eef-2e274ab71e50 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.212698 Oct 04 08:37:20.124576 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 107/215] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:19 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:37:20.168919 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:20.170199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:20.180001 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:20.180703 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:20.181256 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:20.182379 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:20.183063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:20.183771 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:20.266903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:20.281401 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-155af1fe-c871-4aaf-b066-4c2b28a9b4be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1265 microversion: 2.1 time: 0.117068 Oct 04 08:37:20.294785 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 109/216] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:20 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1265 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:20.385004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c81d5f1a-fd03-4db6-a6d5-7f36b111cb2d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: .version_select at 0x7f30a0b36820>, body: {"keypair": {"name": "tempest-keypair-355187951", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHoQby8X8JzOI59Z2QmuBK9LhgtXhL4I4W2hVMCL/Fh5l+iDdrWs3A2VUxTacLeCjzsHnS/QwP/QSAX6q65gOm9Qz/yzZFH8ns2IxmY8Pu7uvyLMm9Dx6nqPjaSnLlft5A=="}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:20.427422 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-c81d5f1a-fd03-4db6-a6d5-7f36b111cb2d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:20.437823 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:20.438394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:20.443478 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-c81d5f1a-fd03-4db6-a6d5-7f36b111cb2d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:20.459419 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:20.460544 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:20.461336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:20.462609 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:20.463214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:20.465997 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:20.487751 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c81d5f1a-fd03-4db6-a6d5-7f36b111cb2d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.106102 Oct 04 08:37:20.489783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 108/217] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:37:20 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:20.602254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:20.602919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:20.607461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:20.613307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:20.614587 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:20.615690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:20.616625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:20.620347 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:20.621334 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:20.629660 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-38f7ed18-323e-4cf4-b06e-9831544a2f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1233 microversion: 2.1 time: 0.194015 Oct 04 08:37:20.643887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 110/218] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:20 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1233 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:20.981410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:20.992628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3b05429e-7069-4bad-938f-0fdf8e22f002 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.399812 Oct 04 08:37:20.993587 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 109/219] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:20 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:21.159333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:21.160031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:21.163122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=5 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:21.174868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:21.175456 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:21.178828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:21.179834 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:21.180326 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:21.180795 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:21.318957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:21.320717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:21.342023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:21.344177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:21.344929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:21.346304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:21.347061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:21.354212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:21.592294 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:21.605830 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-eb68dbdf-28f5-4bea-93ee-808b6c0bb800 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1271 microversion: 2.1 time: 0.301107 Oct 04 08:37:21.607710 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 110/220] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:21 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1271 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:21.656947 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:21.657559 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:21.689226 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:21.691319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:21.691943 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:21.692833 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:21.704286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:21.705028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:22.117787 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:22.119179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:22.128102 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-03b5a8d6-491a-414e-864b-40f61a182c7c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1307 microversion: 2.1 time: 0.475151 Oct 04 08:37:22.128749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 111/221] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:21 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1307 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:22.131376 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:22.134115 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:22.137451 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a172d5f3-38bd-4ac1-b26e-dfef87444834 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.984600 Oct 04 08:37:22.154490 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:22.155501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:22.160743 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 111/222] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:21 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 1001 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:22.167188 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:22.167881 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:22.168582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:22.169829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:22.170299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:22.171047 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:22.563584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:22.572446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b77287fe-f92a-4f6b-84ef-415179e70b3d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.427642 Oct 04 08:37:22.573039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 112/223] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:22 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:22.637717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2051202847", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd380db5-2756-484c-8401-ea63df26e43d"}], "security_groups": [{"name": "tempest-securitygroup--603611596"}], "key_name": "tempest-keypair-355187951", "user_data": "}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:22.640565 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:22.641208 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:22.657464 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:22.657962 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:22.658439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:22.661222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:22.661966 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:22.662787 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:22.927010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] validate_networks() for [('fd380db5-2756-484c-8401-ea63df26e43d', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:37:23.256039 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:23.263858 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2d6a2514-921c-47ec-af72-fec6aa957dae tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 0.635778 Oct 04 08:37:23.275855 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 112/224] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:22 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:23.291484 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:23.292228 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:23.306071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:23.306984 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:23.307646 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:23.308750 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:23.309306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:23.310049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:23.464284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:23.475166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-99c2d80a-088a-412a-8f48-e576b5060c45 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1295 microversion: 2.1 time: 0.198047 Oct 04 08:37:23.475852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 113/225] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:23 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1295 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:23.486052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:23.486957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:23.497340 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:23.498521 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:23.499492 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:23.500363 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:23.501062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:23.501490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:23.507049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:23.988862 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:24.001508 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:24.003547 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:24.004745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c7f019cb-fa87-40e0-a632-9f01dc4102ec tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.527542 Oct 04 08:37:24.005535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 114/226] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:23 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:24.023670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:24.025025 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:24.032403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.033146 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.034696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:24.035859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.036510 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.037149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:24.163287 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:37:24.165468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:37:24.166844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:37:24.167612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:37:24.168265 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:37:24.179045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.180107 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.180776 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:24.182082 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.183013 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.183963 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:24.222099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:24.227991 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:24.246167 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:37:24.249584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:37:24.375658 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-001f4be4-d191-44a6-bb55-01ff99d11ff4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.740278 Oct 04 08:37:24.378972 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 113/227] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:37:22 2022] POST /compute/v2.1/servers => generated 401 bytes in 1748 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 04 08:37:24.397181 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:24.397804 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:24.420416 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.420920 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.421329 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:24.422121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.422772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.423131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:24.447388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:24.463705 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d7d63f05-bedf-42d3-a92e-c3d7bf0784d0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.456690 Oct 04 08:37:24.464174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 115/228] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:24 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:24.485115 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3585742d-2228-437d-ae98-3a7fb761dfb0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:24.489687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3585742d-2228-437d-ae98-3a7fb761dfb0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:24.754878 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3585742d-2228-437d-ae98-3a7fb761dfb0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:24.767678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3585742d-2228-437d-ae98-3a7fb761dfb0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1241 microversion: 2.1 time: 0.300075 Oct 04 08:37:24.770508 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 116/229] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:24 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1241 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:24.787278 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:24.795278 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:24.808683 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.811495 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.812641 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:24.814904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:24.816745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:24.817744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:25.134210 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:25.151810 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f9a00b6a-282b-4a73-84b0-1e67433ca3d1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 0.771367 Oct 04 08:37:25.152479 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 114/230] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:24 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 773 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:25.185047 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-6c55a85b-2c21-4b15-9e3c-33d3e27eaae9 req-8d40ade9-39a2-4ed5-88ec-88e1185ecfd1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a4cf9582-4c11-456d-9fc6-257f9b3dd2c5", "tag": "b38958c1-90c4-4f77-96c2-1d65d20089e8"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:25.198805 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-6c55a85b-2c21-4b15-9e3c-33d3e27eaae9 req-8d40ade9-39a2-4ed5-88ec-88e1185ecfd1 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:25.199652 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-6c55a85b-2c21-4b15-9e3c-33d3e27eaae9 req-8d40ade9-39a2-4ed5-88ec-88e1185ecfd1 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:25.200303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-6c55a85b-2c21-4b15-9e3c-33d3e27eaae9 req-8d40ade9-39a2-4ed5-88ec-88e1185ecfd1 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:25.222374 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-6c55a85b-2c21-4b15-9e3c-33d3e27eaae9 req-8d40ade9-39a2-4ed5-88ec-88e1185ecfd1 service nova] Creating event network-changed:b38958c1-90c4-4f77-96c2-1d65d20089e8 for instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 on hv-cinder-86020 Oct 04 08:37:25.228407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-6c55a85b-2c21-4b15-9e3c-33d3e27eaae9 req-8d40ade9-39a2-4ed5-88ec-88e1185ecfd1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064430 Oct 04 08:37:25.230145 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 115/231] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:25.238052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:25.240081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:25.249657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:25.250213 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:25.250882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:25.253812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:25.254925 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:25.256012 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:25.659017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:25.678275 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-678ad5d3-3bcb-434e-9ce5-ec6e832fedab tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.902584 Oct 04 08:37:25.691216 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 117/232] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:24 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 920 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:25.702020 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:25.702681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:25.714243 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:25.714780 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:25.715461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:25.716222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:25.716840 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:25.717550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:26.037904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:26.051575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:26.054095 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:26.062635 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5d7774f5-e7d8-40c8-8430-df1830156afe tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.830702 Oct 04 08:37:26.063271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 116/233] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:25 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 832 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:26.069011 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:26.069797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:26.084695 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:26.085889 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:26.086398 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:26.087372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:26.087830 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:26.088341 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:26.114950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:26.133782 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a6fe2448-7e7b-444b-bda1-3c7e5a7d84d5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.440293 Oct 04 08:37:26.134612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 118/234] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:25 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:26.179358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:26.179892 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:26.186849 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:26.187893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:26.191290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:26.192373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:26.192809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:26.193273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:26.224993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:26.240173 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1abc66dc-d20f-42c7-9866-48cac7a0b52a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1293 microversion: 2.1 time: 0.175754 Oct 04 08:37:26.241261 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 117/235] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:26 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1293 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:26.613925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:26.633051 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-333b5bc3-e7b3-46a1-8cc8-4deadf3d31f0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 0.458929 Oct 04 08:37:26.636662 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 119/236] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:26 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:26.709623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:26.710389 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:26.724286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:26.724874 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:26.725311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:26.726051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:26.726571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:26.726990 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:26.962195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:26.973699 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0999f00b-0d91-400f-94d6-2090105638bc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.269376 Oct 04 08:37:26.974317 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 118/237] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:26 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:27.093206 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:27.093940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:27.096735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:27.103163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.103650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.104286 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.105145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.105522 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.105909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.169845 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:27.170477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:27.176940 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.177430 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.178139 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.178896 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.179517 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.180304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.311539 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:27.322058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:27.324123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:27.325315 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8657e3f7-953e-446a-9d7b-53c656d7f7ae tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1745 microversion: 2.20 time: 0.236036 Oct 04 08:37:27.325897 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 120/238] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:27 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1745 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:27.330854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:27.331477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:27.339424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.340009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.340506 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.341369 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.341734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.343236 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.365388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:27.375007 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-78883a72-2358-4edd-b5e1-bd48eba5134d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.209165 Oct 04 08:37:27.375585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 119/239] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:27 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:27.381262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-43fea712-b741-4e43-911a-306ca2513746 req-990ad58d-0440-4017-8725-d301fd267dbe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9606a52d-3c5d-4287-afb7-c84f5e245c9a", "tag": "321b30d3-511f-4d46-a3a8-f9853c32c5d9"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:27.391329 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-43fea712-b741-4e43-911a-306ca2513746 req-990ad58d-0440-4017-8725-d301fd267dbe service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.391806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-43fea712-b741-4e43-911a-306ca2513746 req-990ad58d-0440-4017-8725-d301fd267dbe service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.392412 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-43fea712-b741-4e43-911a-306ca2513746 req-990ad58d-0440-4017-8725-d301fd267dbe service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.405958 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-43fea712-b741-4e43-911a-306ca2513746 req-990ad58d-0440-4017-8725-d301fd267dbe service nova] Creating event network-changed:321b30d3-511f-4d46-a3a8-f9853c32c5d9 for instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a on hv-cinder-86020 Oct 04 08:37:27.412270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-43fea712-b741-4e43-911a-306ca2513746 req-990ad58d-0440-4017-8725-d301fd267dbe service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035408 Oct 04 08:37:27.412914 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 120/240] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:27.426227 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:27.439136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2e90ea1b-c947-49cc-a78c-43905bd646c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1299 microversion: 2.1 time: 0.111735 Oct 04 08:37:27.440392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 121/241] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:27 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1299 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:27.663964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:27.664842 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:27.671474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.671951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.672377 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.673065 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:27.673508 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:27.673970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:27.977896 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:27.986080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b30291e6-7b21-499b-8082-a5adee041027 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 0.326191 Oct 04 08:37:27.987336 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 121/242] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:27 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:28.004336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:28.004991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:28.014624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:28.015362 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:28.015965 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:28.016835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:28.017469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:28.018050 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:28.357071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:28.357696 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:28.373582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:28.375171 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:28.375792 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:28.376659 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:28.377976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:28.378762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:28.779918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:28.796214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-82db899c-f3c9-45eb-9308-79a28116b557 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.801688 Oct 04 08:37:28.797169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 122/243] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:27 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 804 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:28.804816 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:28.805641 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:28.812684 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:28.815352 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:28.816069 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:28.816985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:28.817737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:28.819288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:29.056633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:29.078160 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:29.085490 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:29.086875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-dbe8707b-18ff-4853-9e54-f53afc349b9d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1745 microversion: 2.20 time: 0.734752 Oct 04 08:37:29.107028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 122/244] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:28 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1745 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:37:29.107028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:29.107028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:29.107028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:29.123160 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:29.133047 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:29.138065 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:29.139267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:29.141263 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:29.141959 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:29.148433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:29.476782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:29.492845 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8ae33f35-f678-4d5f-87e7-30b251a4c3dc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1299 microversion: 2.1 time: 0.398496 Oct 04 08:37:29.493697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 123/245] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:29 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1299 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:29.513352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:29.513912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:29.526261 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:29.527663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:29.528435 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:29.530321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:29.531707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:29.532858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:29.784507 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:29.793951 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-854a53be-2932-4832-a2ec-d6a639f64fb4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.995234 Oct 04 08:37:29.806851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 123/246] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:28 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 1001 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:29.827937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:29.829599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:29.841932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:29.842974 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:29.843631 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:29.845110 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:29.845965 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:29.847535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:30.223249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:30.233661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-00ca33a4-823a-4d95-a8d6-ce68e0e6596e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 0.738416 Oct 04 08:37:30.234494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 124/247] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:29 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:30.253695 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:30.255600 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:30.273599 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:30.275821 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:30.278279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:30.279287 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:30.280342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:30.282084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:30.473883 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:30.482839 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-61993b76-d739-456b-b9c7-a327ef538d20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.662090 Oct 04 08:37:30.488814 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 124/248] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:29 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:30.520404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:30.521270 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:30.532287 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:30.532949 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:30.536277 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:30.537373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:30.538130 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:30.540578 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:30.852598 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:30.864704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6ca94941-dd65-4cc5-abed-95c1ec335ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1299 microversion: 2.1 time: 0.348631 Oct 04 08:37:30.869688 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 125/249] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:30 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1299 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:30.878428 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-96a98227-5935-40a4-84ad-ad9b10a6b5c2 req-fc71c5ed-d0c2-47a9-8556-c16be212df51 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4cf9582-4c11-456d-9fc6-257f9b3dd2c5", "name": "network-vif-plugged", "status": "completed", "tag": "b38958c1-90c4-4f77-96c2-1d65d20089e8"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:30.906284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-96a98227-5935-40a4-84ad-ad9b10a6b5c2 req-fc71c5ed-d0c2-47a9-8556-c16be212df51 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:30.909589 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-96a98227-5935-40a4-84ad-ad9b10a6b5c2 req-fc71c5ed-d0c2-47a9-8556-c16be212df51 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:30.910195 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-96a98227-5935-40a4-84ad-ad9b10a6b5c2 req-fc71c5ed-d0c2-47a9-8556-c16be212df51 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:30.963068 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-96a98227-5935-40a4-84ad-ad9b10a6b5c2 req-fc71c5ed-d0c2-47a9-8556-c16be212df51 service nova] Creating event network-vif-plugged:b38958c1-90c4-4f77-96c2-1d65d20089e8 for instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 on hv-cinder-86020 Oct 04 08:37:30.991121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-96a98227-5935-40a4-84ad-ad9b10a6b5c2 req-fc71c5ed-d0c2-47a9-8556-c16be212df51 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120103 Oct 04 08:37:30.997717 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 126/250] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:31.013092 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:31.013873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:31.047954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.049822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.064277 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:31.066171 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:31.067636 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.069399 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.074235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:31.086667 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:31.101129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:31.102481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ad484903-c318-4bf2-9fea-020075652ac7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.865259 Oct 04 08:37:31.103142 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 125/251] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:30 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 867 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:31.262193 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:31.262839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:31.279845 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.280340 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.280775 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:31.281486 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.281932 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.282366 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:31.663889 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:31.684186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-450375d8-ed68-4291-841c-0908fa1c358a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.685048 Oct 04 08:37:31.685865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 127/252] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:30 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:31.699919 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:31.712224 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:31.728562 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.729536 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.730048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:31.731111 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.732304 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.733595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:31.733905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:31.743893 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7f3181a0-4f8c-45b9-8a5b-fa62aca7621a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 0.485496 Oct 04 08:37:31.751497 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 126/253] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:31 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:31.912805 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:31.913680 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:31.926590 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.927104 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.927543 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:31.935177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:31.935679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:31.936101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:32.021206 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:32.033865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ebfa605b-ef52-4b1c-a3c3-7a0650fadaa8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.345204 Oct 04 08:37:32.038780 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 128/254] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:31 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:32.145468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:32.146521 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:32.155181 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:32.155939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:32.156694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:32.159744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:32.160525 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:32.161359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:32.287197 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:32.296872 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-abe7f2a7-3eaa-4ed7-b261-49f572dcff5d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 0.389025 Oct 04 08:37:32.297461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 127/255] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:31 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:32.575139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:32.588586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:32.591828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:32.597985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-626340ca-a500-4299-9791-a1a92fe952c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.466884 Oct 04 08:37:32.612631 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 129/256] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:32 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:32.712224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:32.713446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:32.729167 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:32.734153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:32.735062 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:32.737488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:32.739224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:32.740824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:32.781320 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:32.782380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:32.795242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:32.798454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:32.798989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:32.799735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:32.800190 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:32.800586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:33.401256 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:33.414432 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-103e108c-0d80-4f29-85ff-ea7db6eb3e7b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 0.638130 Oct 04 08:37:33.415077 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 130/257] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:32 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:33.428719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:33.429630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:33.445509 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:33.446016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:33.446453 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:33.452530 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:33.453036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:33.453482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:33.575570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:33.594381 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d49214e6-3028-4764-bcf3-29706e7697ef tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.887728 Oct 04 08:37:33.625441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 128/258] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:32 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 920 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:33.640486 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-76fdae91-dd54-4842-92be-0752c5a10c39 req-760b57d6-4de1-4de5-96ea-a838c2f16b46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54819895-e219-4341-8f1e-5543b7357315", "name": "network-vif-plugged", "status": "completed", "tag": "159277a9-e8de-47c4-813b-8de31279c18f"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:33.655969 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-76fdae91-dd54-4842-92be-0752c5a10c39 req-760b57d6-4de1-4de5-96ea-a838c2f16b46 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:33.656679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-76fdae91-dd54-4842-92be-0752c5a10c39 req-760b57d6-4de1-4de5-96ea-a838c2f16b46 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:33.657270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-76fdae91-dd54-4842-92be-0752c5a10c39 req-760b57d6-4de1-4de5-96ea-a838c2f16b46 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:33.678899 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-76fdae91-dd54-4842-92be-0752c5a10c39 req-760b57d6-4de1-4de5-96ea-a838c2f16b46 service nova] Creating event network-vif-plugged:159277a9-e8de-47c4-813b-8de31279c18f for instance 54819895-e219-4341-8f1e-5543b7357315 on hv-cinder-86020 Oct 04 08:37:33.685248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-76fdae91-dd54-4842-92be-0752c5a10c39 req-760b57d6-4de1-4de5-96ea-a838c2f16b46 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052955 Oct 04 08:37:33.686365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 129/259] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:33.703565 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:33.704387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:33.725761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:33.726134 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:33.726623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:33.727284 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:33.727727 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:33.728142 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:34.210344 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=4 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:34.241300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=18 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:34.286710 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:34.298084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-19fae2f4-699b-40da-83b8-dc39e3851d15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.880281 Oct 04 08:37:34.303082 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 131/260] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:33 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 886 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:34.313662 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:34.315694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:34.328183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:34.339702 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:34.340570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7d60fd9b-3545-4f16-9e92-7f828d2ba2db tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 0.647397 Oct 04 08:37:34.340784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:34.341234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:34.342655 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 130/261] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:33 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:34.348044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:34.348553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:34.357335 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:34.420428 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:34.445670 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:34.448559 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:34.491413 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:34.491413 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:34.491413 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:34.496523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:34.497211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:34.499984 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:34.767143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=18 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:34.843719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:34.874383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:34.875899 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:34.878044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7d89deb7-bee3-4e0d-a001-c539c825f469 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.573440 Oct 04 08:37:34.879153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 132/262] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:34 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:34.890076 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-9e902129-4bb5-433f-a86b-d50c1144d2e2 req-e48b4ea1-4ac8-4815-a3b9-4cae580d0540 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6dee148f-e8b5-437c-9d80-4d784091314c", "tag": "ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:34.911551 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e902129-4bb5-433f-a86b-d50c1144d2e2 req-e48b4ea1-4ac8-4815-a3b9-4cae580d0540 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:34.912034 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e902129-4bb5-433f-a86b-d50c1144d2e2 req-e48b4ea1-4ac8-4815-a3b9-4cae580d0540 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:34.912465 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e902129-4bb5-433f-a86b-d50c1144d2e2 req-e48b4ea1-4ac8-4815-a3b9-4cae580d0540 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:34.950922 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-9e902129-4bb5-433f-a86b-d50c1144d2e2 req-e48b4ea1-4ac8-4815-a3b9-4cae580d0540 service nova] Creating event network-changed:ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c for instance 6dee148f-e8b5-437c-9d80-4d784091314c on hv-cinder-86020 Oct 04 08:37:34.966924 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-9e902129-4bb5-433f-a86b-d50c1144d2e2 req-e48b4ea1-4ac8-4815-a3b9-4cae580d0540 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086329 Oct 04 08:37:34.967916 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 133/263] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:34.981843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:34.982586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:34.995864 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:34.996353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:34.996788 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:34.997496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:34.997947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:34.998368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:35.810256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:35.823662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-02d7da47-4cef-4be6-a60f-fe97acb959e3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1313 microversion: 2.1 time: 1.383262 Oct 04 08:37:35.827003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 131/264] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:34 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1313 bytes in 1388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:35.833912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:35.841530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:35.864893 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:35.865799 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:35.868245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:35.869630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:35.871092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:35.873576 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:35.955937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:35.967270 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c2691573-1a96-4589-8c9f-bd579f438d22 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.997339 Oct 04 08:37:35.969654 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 134/265] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:34 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 1001 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:35.978933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:35.981347 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:36.024461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:36.030957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:36.034863 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:36.035941 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:36.037410 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:36.043266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:36.463728 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:36.472624 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7c23ae16-f77d-4fe6-b34e-d5d2836755ba tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.644254 Oct 04 08:37:36.473216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 132/266] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:35 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:36.483253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-2017588242", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2095819453", "security_groups": [{"name": "tempest-secgroup-smoke-1159980666"}], "block_device_mapping_v2": [{"uuid": "1478f59f-9b4d-4eb0-827a-f36c53d04641", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e71b68ee-781f-4e9d-8cf4-0cfc8be1f31a"}]}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:36.512892 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:36.527536 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f296eb26-6003-49c1-b85d-d79ea595c97d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 0.549868 Oct 04 08:37:36.528497 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 135/267] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:35 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:36.542480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:36.543438 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:36.557235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:36.557737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:36.558171 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:36.558912 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:36.559818 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:36.560251 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:36.782419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] validate_networks() for [('e71b68ee-781f-4e9d-8cf4-0cfc8be1f31a', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:37:36.880838 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:36.897139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:36.900082 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:36.902581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4c1499aa-15dd-4f8a-934b-dd4b538b3c71 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.372914 Oct 04 08:37:36.932159 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 136/268] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:36 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:36.944753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:36.947687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:36.957581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:36.958242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:36.958494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:36.959429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:36.960135 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:36.960890 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:37.262031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:37.272675 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7dd4357a-c993-47bf-813a-1a150c48521e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1495 microversion: 2.1 time: 0.339095 Oct 04 08:37:37.273819 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 137/269] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:36 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1495 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:37.279119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:37.279971 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:37.290827 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:37.293603 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:37.294976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:37.297562 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:37.298336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:37.300627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:37.627737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:37.637188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1dce5c31-479d-47cd-8e1d-b089ede6f510 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1502 microversion: 2.1 time: 0.362128 Oct 04 08:37:37.637781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 138/270] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:37 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1502 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:37.648752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:37.649611 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:37.656908 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:37.657640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:37.659430 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:37.660359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:37.660979 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:37.661738 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:37.827231 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:37:37.827706 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:37:37.828016 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:37:37.828424 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:37:37.829006 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:37:37.844502 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:37.845663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:37.846300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:37.847558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:37.848256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:37.848854 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:37.889351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Getting quotas for project 2f94e3ae140b43739febe1111e18604b. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:37.895749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Getting quotas for user ab4b3c2b3f4b4978b0eb472f9c2736b1 and project 2f94e3ae140b43739febe1111e18604b. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:38.007812 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:38.016557 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:37:38.019552 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='1478f59f-9b4d-4eb0-827a-f36c53d04641',volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:37:38.025095 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-efc0ff41-035f-4fcf-9c38-663b8bfd7336 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.386260 Oct 04 08:37:38.027071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 139/271] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:37 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:38.039604 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:38.041307 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:38.047926 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.049208 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.049837 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.050872 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.051333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.051765 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.291067 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9b773d4f-9f89-4213-a5e3-17e3d42019f9 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.811468 Oct 04 08:37:38.292525 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 133/272] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:37:36 2022] POST /compute/v2.1/servers => generated 406 bytes in 1813 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:37:38.299761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "26a2f028-b2d2-489d-9498-a1e5d409956b"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:38.302843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:38.310961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.311936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.313330 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.314991 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.315833 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.316428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.348407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Cannot 'attach_volume' instance f597e06f-510c-46fe-92de-ce4a8dc53841 while it is in vm_state rescued Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: POST /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841/os-volume_attachments HTTP/1.1 Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 74 Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:37:38.352232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {"volumeAttachment": {"volumeId": "26a2f028-b2d2-489d-9498-a1e5d409956b"}} Oct 04 08:37:38.353742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Cannot 'attach_volume' instance f597e06f-510c-46fe-92de-ce4a8dc53841 while it is in vm_state rescued Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f597e06f-510c-46fe-92de-ce4a8dc53841', 'body': {'volumeAttachment': {'volumeId': '26a2f028-b2d2-489d-9498-a1e5d409956b'}}} Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance f597e06f-510c-46fe-92de-ce4a8dc53841 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:37:38.354630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance f597e06f-510c-46fe-92de-ce4a8dc53841 while it is in vm_state rescued Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:37:38.355917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:37:38.357028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:37:38.357028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:37:38.357028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance f597e06f-510c-46fe-92de-ce4a8dc53841 while it is in vm_state rescued Oct 04 08:37:38.357028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:38.358371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 409 to user: Cannot 'attach_volume' instance f597e06f-510c-46fe-92de-ce4a8dc53841 while it is in vm_state rescued {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:37:38.359510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d560d40c-299d-469d-ab05-243903cfab29 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.065805 Oct 04 08:37:38.360633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 134/273] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:37:38 2022] POST /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841/os-volume_attachments => generated 152 bytes in 67 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 04 08:37:38.365981 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:38.366892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:38.381311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.381929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:38.382463 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.383702 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.384959 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.385911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.386639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.392309 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6a8bf766-7279-47d5-91bf-e3f6df0fa29f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 0.363900 Oct 04 08:37:38.393148 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 140/274] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:38 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:38.398604 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3d2b67cd-7866-4918-a4cb-bc5de43ddd0d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:38.399809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3d2b67cd-7866-4918-a4cb-bc5de43ddd0d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:38.547681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3d2b67cd-7866-4918-a4cb-bc5de43ddd0d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:38.558476 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3d2b67cd-7866-4918-a4cb-bc5de43ddd0d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1095 microversion: 2.1 time: 0.163757 Oct 04 08:37:38.562028 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 141/275] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:38 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1095 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:38.570081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:38.583027 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:38.598418 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.599719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.602372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.604216 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.607427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.609004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.773018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:38.789740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:38.804563 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-dec36085-9b07-4acb-9c26-7a9782a0d7b4 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841/action" status: 202 len: 0 microversion: 2.1 time: 0.240229 Oct 04 08:37:38.807087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 142/276] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:37:38 2022] POST /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841/action => generated 0 bytes in 244 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:37:38.808042 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:38.809320 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a434818a-56f3-4e00-9db4-bf1295b99d15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.447342 Oct 04 08:37:38.809986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 135/277] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:38 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:38.827006 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:38.827703 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:38.842368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.843031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.843672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:38.844616 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:38.845921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:38.847922 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:39.057700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:39.058942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:39.068113 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:39.068862 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:39.069754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:39.070656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:39.071269 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:39.072533 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:39.386136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:39.396012 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3ce892d5-ec95-4268-ae12-373b808e0851 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1510 microversion: 2.1 time: 0.583901 Oct 04 08:37:39.401053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 143/278] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:38 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1510 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:39.416659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:39.417640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:39.433089 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:39.433968 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:39.434723 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:39.435670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:39.436317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:39.436930 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:39.711648 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:39.725877 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-80a01d13-e9fb-4904-b525-e26b099d6f7f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 0.672645 Oct 04 08:37:39.733705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 136/279] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:39 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:39.738940 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:39.745748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:39.762095 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:39.762610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:39.763048 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:39.763755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:39.764210 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:39.764660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:39.821300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:37:40.013840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-25daeb48-4ec0-478b-970e-e755dcc634e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 204 len: 0 microversion: 2.1 time: 0.278555 Oct 04 08:37:40.015047 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 137/280] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:37:39 2022] DELETE /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:37:40.022617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:40.023762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:40.031115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:40.032146 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:40.032902 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:40.034074 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:40.038477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:40.039240 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:40.275355 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:40.288972 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cf350eeb-f40b-41d5-bb25-ddd82371e321 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1193 microversion: 2.1 time: 0.270144 Oct 04 08:37:40.291660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 138/281] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:40 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1193 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:40.313691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:40.315386 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:40.323051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:40.323735 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:40.324337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:40.325187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:40.325825 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:40.326435 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:40.535166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:40.550414 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bebffe5a-0f06-4492-99a7-d166c59a4dac tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 1.137964 Oct 04 08:37:40.551049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 144/282] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:39 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 1139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:40.562766 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:40.566370 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:40.583066 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:40.584120 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:40.585835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:40.587323 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:40.587777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:40.591682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:40.787621 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:40.789926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:40.805825 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:40.819756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:40.821149 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6c1a769a-e96b-46d8-9cf3-c6baf6ccb914 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.514395 Oct 04 08:37:40.821999 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 139/283] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:40 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:40.826490 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-f05382a1-6044-4b50-8880-019921e10c0c req-17e11f8e-63ac-411f-8500-06e5e9f54ba4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9606a52d-3c5d-4287-afb7-c84f5e245c9a", "name": "network-vif-plugged", "status": "completed", "tag": "321b30d3-511f-4d46-a3a8-f9853c32c5d9"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:40.857904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f05382a1-6044-4b50-8880-019921e10c0c req-17e11f8e-63ac-411f-8500-06e5e9f54ba4 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:40.858386 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f05382a1-6044-4b50-8880-019921e10c0c req-17e11f8e-63ac-411f-8500-06e5e9f54ba4 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:40.858902 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f05382a1-6044-4b50-8880-019921e10c0c req-17e11f8e-63ac-411f-8500-06e5e9f54ba4 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:40.978166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:40.988994 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-855ff80d-7e77-4730-9452-124bcba524e0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1501 microversion: 2.1 time: 0.430087 Oct 04 08:37:40.995756 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 145/284] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:40 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1501 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:41.004413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:41.005827 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:41.023542 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:41.023976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:41.024421 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:41.025188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:41.025652 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:41.026153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:41.199832 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-f05382a1-6044-4b50-8880-019921e10c0c req-17e11f8e-63ac-411f-8500-06e5e9f54ba4 service nova] Creating event network-vif-plugged:321b30d3-511f-4d46-a3a8-f9853c32c5d9 for instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a on hv-cinder-86020 Oct 04 08:37:41.214515 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-f05382a1-6044-4b50-8880-019921e10c0c req-17e11f8e-63ac-411f-8500-06e5e9f54ba4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.391133 Oct 04 08:37:41.215762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 140/285] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:41.231754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:41.233904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:41.243120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:41.243918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:41.244730 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:41.245744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:41.246783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:41.247433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:41.947267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:41.974584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-177eae9a-3bce-4568-bb33-4d438d9c2f69 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1510 microversion: 2.1 time: 0.977384 Oct 04 08:37:41.982220 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 146/286] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:40 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1510 bytes in 985 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:42.002898 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:42.010579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:42.017458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:42.018019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:42.018425 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:42.019198 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:42.032223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:42.033757 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:42.259888 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:42.273563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:42.275432 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3097aa4b-e08b-4fa0-a75a-bef2807f3a6a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1193 microversion: 2.1 time: 0.287936 Oct 04 08:37:42.277646 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 147/287] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:41 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1193 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:42.285230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:42.286380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:42.290006 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cb6da8b1-9d74-4a30-8d28-f7f8463e0af0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1363 microversion: 2.1 time: 1.064763 Oct 04 08:37:42.306669 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:42.312292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:42.312613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 141/288] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:41 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1363 bytes in 1096 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:42.315495 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:42.316505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:42.317644 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:42.319149 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:42.319851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:42.320295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:42.339648 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:42.340459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:42.341119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:42.342042 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:42.342735 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:42.343382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:43.147177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:43.164530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:43.167766 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:43.169774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e8bb58bd-573b-43e4-834f-60dea0d786b4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.854010 Oct 04 08:37:43.176957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 142/289] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:42 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:43.184886 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:43.187027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:43.195873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:43.197077 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:43.198839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:43.200428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:43.201213 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:43.201888 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:43.205765 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:43.220186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2f78b7d9-b327-468c-8ce6-bb2425714a5a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 0.939393 Oct 04 08:37:43.242932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 148/290] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:42 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 962 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:43.265541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:43.266677 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:43.275189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:43.275950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:43.276455 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:43.277270 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:43.277781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:43.278424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:43.864993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:43.890895 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c230f748-e150-44f5-9b8f-c79a7080c3be tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1501 microversion: 2.1 time: 0.708562 Oct 04 08:37:43.891368 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 143/291] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:43 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1501 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:43.906158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:43.907255 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:43.917234 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:43.919742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:43.922630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:43.925647 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:43.927409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:43.929731 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:44.281074 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:44.307022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d6af1f22-ca70-4fb7-b247-25c81f381202 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1211 microversion: 2.1 time: 0.410744 Oct 04 08:37:44.318594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 144/292] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:43 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1211 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:44.328633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:44.334719 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:44.355154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:44.363494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:44.365819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:44.367782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:44.368812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:44.370627 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:44.648393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:44.651037 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:44.662364 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-fba58793-9d5a-4334-b36d-2c9376211d6e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1510 microversion: 2.1 time: 1.417555 Oct 04 08:37:44.666970 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 149/293] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:43 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1510 bytes in 1423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:44.678416 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:44.698996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:44.706182 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:44.707118 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:44.707728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:44.708553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:44.709024 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:44.709490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:45.877222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:45.884027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:45.893417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-82fdcb5f-801d-4402-891f-3c2a29295196 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1550 microversion: 2.1 time: 1.583340 Oct 04 08:37:45.900929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:45.906967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:45.908620 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-64a5489e-0547-4804-b7db-b38ae989a1d9 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 1.240393 Oct 04 08:37:45.909389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 150/294] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:44 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 1242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:45.914519 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 145/295] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:44 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1550 bytes in 1602 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:45.947491 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:45.950091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:45.951517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:45.953333 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:45.981496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:45.988377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:45.991792 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:45.992310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:45.992778 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:45.993246 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:45.993671 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:45.994485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:45.995663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:45.997138 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:45.998151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:45.999419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.065744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.067222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.068438 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.069605 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lazy-loading 'fault' on Instance uuid a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:37:46.251154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:46.284453 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5a0366c0-b3fa-4e6e-8553-4768a7d312d6 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1303 microversion: 2.1 time: 0.364235 Oct 04 08:37:46.303057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 146/296] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:45 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1303 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:46.309920 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-7694fe29-71f9-4d8c-b5ba-66052b7404c6 req-7f220752-a62f-4ed6-a2c0-1434478d8c41 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4cf9582-4c11-456d-9fc6-257f9b3dd2c5", "name": "network-vif-deleted", "tag": "b38958c1-90c4-4f77-96c2-1d65d20089e8"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:46.356658 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-7694fe29-71f9-4d8c-b5ba-66052b7404c6 req-7f220752-a62f-4ed6-a2c0-1434478d8c41 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.359953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-7694fe29-71f9-4d8c-b5ba-66052b7404c6 req-7f220752-a62f-4ed6-a2c0-1434478d8c41 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.360389 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-7694fe29-71f9-4d8c-b5ba-66052b7404c6 req-7f220752-a62f-4ed6-a2c0-1434478d8c41 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.407454 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-7694fe29-71f9-4d8c-b5ba-66052b7404c6 req-7f220752-a62f-4ed6-a2c0-1434478d8c41 service nova] Creating event network-vif-deleted:b38958c1-90c4-4f77-96c2-1d65d20089e8 for instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 on hv-cinder-86020 Oct 04 08:37:46.429429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-7694fe29-71f9-4d8c-b5ba-66052b7404c6 req-7f220752-a62f-4ed6-a2c0-1434478d8c41 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.119335 Oct 04 08:37:46.433744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 147/297] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:46.441339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:46.442098 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:46.449014 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:46.450179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.450716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.451154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.452441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.452929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.453417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.467358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-17a9797b-e744-40a0-981b-0e78bcde6f0f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 0.547280 Oct 04 08:37:46.480198 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 151/298] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:45 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:46.494197 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:46.495480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:46.504792 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.507420 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.508776 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.512102 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.513486 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.514165 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.689287 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:46.700963 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-463ce402-21a0-405c-af7b-5e31223a76ff tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1199 microversion: 2.1 time: 0.263643 Oct 04 08:37:46.714589 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 148/299] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:46 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1199 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:46.730434 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:46.730983 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:46.751663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.752162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.752617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.753305 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:46.753761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:46.754253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:46.964383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:46.975403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-da882729-af0c-4ba9-a2b8-c56171071350 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1510 microversion: 2.1 time: 0.491368 Oct 04 08:37:46.986953 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 152/300] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:46 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1510 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:46.992338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:46.993209 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:47.001712 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:47.001889 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f6b9a862-b79e-4d84-b754-05f2f5f47cc7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.287623 Oct 04 08:37:47.003152 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:47.005132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:47.006078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:47.008873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:47.009522 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:47.010005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 149/301] 10.222.0.16 () {58 vars in 1275 bytes} [Tue Oct 4 08:37:46 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-interface => generated 285 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:47.311582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:47.312214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:47.322047 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:47.322578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:47.323027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:47.334511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:47.337674 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:47.342118 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:47.450264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:47.451144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:47.455892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:47.465809 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lazy-loading 'fault' on Instance uuid a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:37:47.836219 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:47.849717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c0fe61c5-5cdc-4ed2-9a28-3a3f41e23350 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 200 len: 1303 microversion: 2.1 time: 0.542676 Oct 04 08:37:47.862705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 150/302] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:47 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 1303 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:47.870949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:47.871994 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:47.889522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:47.900726 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:47.901237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:47.901701 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:47.902421 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:47.908640 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:47.908640 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:47.909355 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=6 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:48.102230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:48.141189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:48.145374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:48.148199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-09279ba8-953c-42a3-9b6b-0663cc3d48b3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 1.159722 Oct 04 08:37:48.175472 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 153/303] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:46 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 1188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:48.193884 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:48.194689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:48.212884 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:48.213689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:48.214313 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:48.218418 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:48.219052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:48.219638 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:48.552768 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.636702 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:49.652819 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-306c7ee6-b7d9-49b4-8056-a6ac58da7b01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 1.475856 Oct 04 08:37:49.655494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 154/304] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:48 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 1479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:49.671658 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:49.672777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:49.685625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:49.686147 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:49.686615 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:49.687317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:49.687763 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:49.688184 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:49.907245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:49.933505 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9af1f9d3-9762-407f-8102-93a5ffc31344 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 2.081854 Oct 04 08:37:49.939824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 151/305] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:47 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 2089 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:49.965642 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-02a40d3c-4cd2-43a5-b560-03f78a14846c req-fe60917a-b737-4483-96e8-b587651335b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6dee148f-e8b5-437c-9d80-4d784091314c", "name": "network-vif-plugged", "status": "completed", "tag": "ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:49.997471 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-02a40d3c-4cd2-43a5-b560-03f78a14846c req-fe60917a-b737-4483-96e8-b587651335b1 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:50.001926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-02a40d3c-4cd2-43a5-b560-03f78a14846c req-fe60917a-b737-4483-96e8-b587651335b1 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:50.002604 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-02a40d3c-4cd2-43a5-b560-03f78a14846c req-fe60917a-b737-4483-96e8-b587651335b1 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:50.038151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-02a40d3c-4cd2-43a5-b560-03f78a14846c req-fe60917a-b737-4483-96e8-b587651335b1 service nova] Creating event network-vif-plugged:ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c for instance 6dee148f-e8b5-437c-9d80-4d784091314c on hv-cinder-86020 Oct 04 08:37:50.047388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-02a40d3c-4cd2-43a5-b560-03f78a14846c req-fe60917a-b737-4483-96e8-b587651335b1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091233 Oct 04 08:37:50.048633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 152/306] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:50.056325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:50.062522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: a4cf9582-4c11-456d-9fc6-257f9b3dd2c5] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:50.086306 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:50.087145 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:50.087758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:50.090635 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:50.092024 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:50.094021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:50.159796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] HTTP exception thrown: Instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 could not be found. Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Request method failure captured: Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 HTTP/1.1 Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_C4Ow5MXLpWaNKHXTM5mvowxqPB7vq1v1xGYxH5rzF3OukQlkrenYbx2TbY7ji0A3328RRzx67kO6Ym263GPs0VGBWhhKTT8WXtlpdgiv3HDkSIwRSu08NMfFPWnqg-u9PQ799I2hoBYABbnECkvOijbfVVtjLXlg2nK7CmFtKIp44 Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f758e7615232486a93a915760633b702 Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesActionsTest-1354781618 Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:37:50.161716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f758e7615232486a93a915760633b702"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f758e7615232486a93a915760633b702"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f758e7615232486a93a915760633b702"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f758e7615232486a93a915760633b702", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesActionsTest-1354781618 Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f758e7615232486a93a915760633b702 Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesActionsTest-1354781618 Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesActionsTest-1354781618-project Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 76b2078f1f64440e9d8faca3bcf98d52 Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesActionsTest-1354781618-project Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 could not be found. Oct 04 08:37:50.163469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'a4cf9582-4c11-456d-9fc6-257f9b3dd2c5'} Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 could not be found. Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:37:50.176922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 could not be found. Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 could not be found. Oct 04 08:37:50.180274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:37:50.182572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Returning 404 to user: Instance a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:37:50.182572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4ac69492-8134-4da4-838f-e2f83856f197 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5" status: 404 len: 111 microversion: 2.1 time: 0.113267 Oct 04 08:37:50.182572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 153/307] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:50 2022] GET /compute/v2.1/servers/a4cf9582-4c11-456d-9fc6-257f9b3dd2c5 => generated 111 bytes in 130 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:37:50.205633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:50.206326 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:50.243323 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:50.243817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:50.244255 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:50.244981 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:50.245441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:50.245861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:50.802347 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:50.820815 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6eb52f10-2929-42c1-a427-25696b6ce5a7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1516 microversion: 2.1 time: 1.157603 Oct 04 08:37:50.822190 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 155/308] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:49 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1516 bytes in 1166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:50.831829 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-95439376", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "networks": [{"uuid": "1ee6d49b-e78d-48c2-9f17-a27346795ae3"}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:50.964448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] validate_networks() for [('1ee6d49b-e78d-48c2-9f17-a27346795ae3', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:37:51.774539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:51.797119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:51.800544 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:51.802390 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-41316108-59ff-401c-8d6e-881202028929 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 1.605838 Oct 04 08:37:51.804289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 154/309] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:50 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 1624 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:51.817572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:51.819902 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:51.838057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:51.839607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:51.840858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:51.841992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:51.843143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:51.843956 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:52.589037 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:52.610277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8364a8fb-d41a-4b13-ae24-a00d25f5866d tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.797526 Oct 04 08:37:52.612993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 155/310] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:51 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 808 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:52.639562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:52.640859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:52.666035 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:52.677464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:52.684508 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:52.689534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:52.689534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:52.690005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:52.980108 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:37:52.980931 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:37:52.981290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:37:52.982833 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:37:52.983386 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:37:52.990568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=96 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:53.023637 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:53.024722 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:53.028392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:53.032263 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:53.037699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:53.053221 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:53.130166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Getting quotas for project f758e7615232486a93a915760633b702. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:53.169656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Getting quotas for user 76b2078f1f64440e9d8faca3bcf98d52 and project f758e7615232486a93a915760633b702. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:53.217351 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:37:53.226490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:37:53.452427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e7e4bd23-c7d0-487e-95e1-96daa8286b2a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.628874 Oct 04 08:37:53.464845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 156/311] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:37:50 2022] POST /compute/v2.1/servers => generated 380 bytes in 2642 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:37:53.494200 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:53.500704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:53.520063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:53.526996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:53.528477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:53.531366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:53.532122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:53.533374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:53.958342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:53.967837 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fcbaf368-9811-45a0-a789-6a92ecdaaf9a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1366 microversion: 2.1 time: 1.353113 Oct 04 08:37:53.982392 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 156/312] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:52 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1366 bytes in 1369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:53.998452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c1217912-286f-4aaa-af46-f324581335f7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'create', calling method: .version_select at 0x7f30a0c973a0>, body: {"keypair": {"name": "tempest-keypair-2108872820", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBObZVQhCU39mhlG2gwyBwrGIDYGami853Qk0SZuA29RxPUofwvLnL800AsJHyUFoFpJP1UehBXQMXSJWoO6WUsBBWLSSIVvHyFguPhCDFz71vvuqvW1btkXTqJZuMebpew=="}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:54.101963 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-c1217912-286f-4aaa-af46-f324581335f7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for project 5934621267af4b838889fcbfb9febaea. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:54.120227 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-c1217912-286f-4aaa-af46-f324581335f7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for user 26abb94392364bdc82f6bb508a611d44 and project 5934621267af4b838889fcbfb9febaea. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:54.154356 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:54.279173 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c1217912-286f-4aaa-af46-f324581335f7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.286336 Oct 04 08:37:54.283987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 157/313] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:37:53 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:54.316101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f30d4bfb-123d-4624-bb19-6345f36f4a02 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:54.320175 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f30d4bfb-123d-4624-bb19-6345f36f4a02 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:54.337042 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:54.364172 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:54.369061 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:54.371265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-056aced1-24e8-41fe-8589-aa0b5b60dbc2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 0.905121 Oct 04 08:37:54.372769 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 157/314] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:53 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 907 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:54.380619 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:54.381491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:54.409531 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:54.410289 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:54.410931 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:54.411833 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:54.412456 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:54.413221 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:54.713351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f30d4bfb-123d-4624-bb19-6345f36f4a02 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:54.726819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f30d4bfb-123d-4624-bb19-6345f36f4a02 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1212 microversion: 2.1 time: 0.438324 Oct 04 08:37:54.733063 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 158/315] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:54 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1212 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:54.785094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-fecd88e6-5e83-46aa-94ed-4c4ec3bb2623 req-f6ddcd73-8dee-4cda-9742-fd6ad86a2b70 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c1f2b98d-4c4e-47ff-b1d4-19addcee2e27", "tag": "0e892634-2abc-4cc1-ad5b-83887511e34a"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:54.832765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-fecd88e6-5e83-46aa-94ed-4c4ec3bb2623 req-f6ddcd73-8dee-4cda-9742-fd6ad86a2b70 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:54.834342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-fecd88e6-5e83-46aa-94ed-4c4ec3bb2623 req-f6ddcd73-8dee-4cda-9742-fd6ad86a2b70 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:54.839142 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-fecd88e6-5e83-46aa-94ed-4c4ec3bb2623 req-f6ddcd73-8dee-4cda-9742-fd6ad86a2b70 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:54.872688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-fecd88e6-5e83-46aa-94ed-4c4ec3bb2623 req-f6ddcd73-8dee-4cda-9742-fd6ad86a2b70 service nova] Creating event network-changed:0e892634-2abc-4cc1-ad5b-83887511e34a for instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 on hv-cinder-86020 Oct 04 08:37:54.882711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-fecd88e6-5e83-46aa-94ed-4c4ec3bb2623 req-f6ddcd73-8dee-4cda-9742-fd6ad86a2b70 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.125241 Oct 04 08:37:54.887123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 159/316] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:55.019899 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:55.020776 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:55.041895 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:55.043076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:55.046084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:55.049506 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:55.050914 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:55.051758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:55.071405 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:55.083859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d11ad428-5916-4f61-8538-f17c08fe927f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.709575 Oct 04 08:37:55.085043 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 158/317] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:54 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 711 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:55.403746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:55.405109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:55.421671 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:55.425249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:55.429359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:55.431491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:55.434218 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:55.438287 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:55.581901 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:55.608264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8df45c50-a2c2-4e03-9896-d0558eb63f5f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1556 microversion: 2.1 time: 0.599784 Oct 04 08:37:55.623317 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 160/318] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:55 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1556 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:37:55.636135 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:55.637012 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:55.645269 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:55.646178 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:55.647052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:55.647943 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:55.648625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:55.649574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:55.682628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:56.088183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2488eab3-7008-4811-b09e-0169e4f03f40 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.457438 Oct 04 08:37:56.097472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 161/319] 10.222.0.16 () {58 vars in 1275 bytes} [Tue Oct 4 08:37:55 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c/os-interface => generated 284 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:37:56.102619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:56.103714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:56.118260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:56.119054 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:56.119746 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:56.120625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:56.121283 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:56.125122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:56.541130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:56.549899 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2d684187-f48e-4a17-8fa1-7fe361bf6d26 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1264 microversion: 2.1 time: 0.451022 Oct 04 08:37:56.563369 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 162/320] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:56 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1264 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:56.568225 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:56.568834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:56.575178 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:56.575774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:56.576260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:56.580563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:56.583744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:56.584390 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:56.676183 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:56.700447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:56.703458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:56.705604 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-374b3c0f-29f8-4eb2-b5ca-68740ef00705 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1740 microversion: 2.20 time: 1.305738 Oct 04 08:37:56.724170 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 159/321] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:55 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1740 bytes in 1325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:57.118049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-f9de6e55-f1fc-4f76-ad5e-5ea8ae377df2 req-047280bf-d3ea-41a8-a7fe-43448c0a89bb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9606a52d-3c5d-4287-afb7-c84f5e245c9a", "tag": "321b30d3-511f-4d46-a3a8-f9853c32c5d9"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:57.149806 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f9de6e55-f1fc-4f76-ad5e-5ea8ae377df2 req-047280bf-d3ea-41a8-a7fe-43448c0a89bb service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:57.152533 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f9de6e55-f1fc-4f76-ad5e-5ea8ae377df2 req-047280bf-d3ea-41a8-a7fe-43448c0a89bb service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:57.153992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f9de6e55-f1fc-4f76-ad5e-5ea8ae377df2 req-047280bf-d3ea-41a8-a7fe-43448c0a89bb service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:57.209441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-f9de6e55-f1fc-4f76-ad5e-5ea8ae377df2 req-047280bf-d3ea-41a8-a7fe-43448c0a89bb service nova] Creating event network-changed:321b30d3-511f-4d46-a3a8-f9853c32c5d9 for instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a on hv-cinder-86020 Oct 04 08:37:57.217004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-f9de6e55-f1fc-4f76-ad5e-5ea8ae377df2 req-047280bf-d3ea-41a8-a7fe-43448c0a89bb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.107572 Oct 04 08:37:57.218205 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 160/322] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:57.619049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:57.621195 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:57.636163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:57.640099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:57.641288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:57.646447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:57.647667 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:57.649876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:57.994886 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:58.015215 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4f9f2e46-0e02-4a91-8aed-c46c66f46efa tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1270 microversion: 2.1 time: 0.421790 Oct 04 08:37:58.018745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 161/323] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:57 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1270 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:58.033325 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:58.035297 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:58.047720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:58.049835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:58.051786 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:58.053664 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:58.054828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:58.056341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:58.074614 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:58.108322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8a66d0ff-ac9d-4a21-9c3a-0af259ddab1c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 1.537481 Oct 04 08:37:58.109865 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 163/324] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:56 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 1546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:59.145966 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:59.160979 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:59.167663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:59.169682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b29f5318-4d22-4274-bac1-d1fb8021b31f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1722 microversion: 2.20 time: 1.144855 Oct 04 08:37:59.192322 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 162/325] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:37:58 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1722 bytes in 1172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:37:59.206694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:59.207596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:59.229079 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d9e99e1a-0631-41a7-b863-e55ab88488c5 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Action: 'create', calling method: .version_select at 0x7f30a0de7670>, body: {"keypair": {"name": "tempest-TestShelveInstance-391951627", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG8k+haFkNoXsxyA2dqCObxrJ6MPW9n/zPwMIQyXyFqPtskQVGT4C27SpuPAJl0J3dUrHQyQeYBhXeFQqW4fRIaO4Jmj0J8Gg4QINEa6cNHNX3qykDGJjlDBKfgSLDQOLQ=="}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:59.242818 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:59.243653 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:59.244411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:59.246022 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:59.246842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:59.247477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:59.268034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-d9e99e1a-0631-41a7-b863-e55ab88488c5 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Getting quotas for project ed407864f9cf431ebd8018e1a6862294. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:37:59.274188 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-d9e99e1a-0631-41a7-b863-e55ab88488c5 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Getting quotas for user d1520d0cbea447a3a44a992ca10ca587 and project ed407864f9cf431ebd8018e1a6862294. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:37:59.328065 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d9e99e1a-0631-41a7-b863-e55ab88488c5 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.507787 Oct 04 08:37:59.337141 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 164/326] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:37:58 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:37:59.350897 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:37:59.351986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:37:59.361441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:59.362331 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:59.363050 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:59.364176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:59.364927 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:59.365528 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:59.459807 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:37:59.471164 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-96b0a952-108b-4ba4-b449-b52af508c186 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1270 microversion: 2.1 time: 0.274204 Oct 04 08:37:59.472882 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 163/327] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:59 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1270 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:37:59.691161 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-68d06b36-3f11-4fec-8bfc-b6b472364edb req-aacce309-1df0-4946-a1c0-5d2e5d7264d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54819895-e219-4341-8f1e-5543b7357315", "name": "network-vif-unplugged", "status": "completed", "tag": "159277a9-e8de-47c4-813b-8de31279c18f"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:37:59.714219 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-68d06b36-3f11-4fec-8bfc-b6b472364edb req-aacce309-1df0-4946-a1c0-5d2e5d7264d6 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:37:59.714753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-68d06b36-3f11-4fec-8bfc-b6b472364edb req-aacce309-1df0-4946-a1c0-5d2e5d7264d6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:37:59.715485 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-68d06b36-3f11-4fec-8bfc-b6b472364edb req-aacce309-1df0-4946-a1c0-5d2e5d7264d6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:37:59.733394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-68d06b36-3f11-4fec-8bfc-b6b472364edb req-aacce309-1df0-4946-a1c0-5d2e5d7264d6 service nova] Creating event network-vif-unplugged:159277a9-e8de-47c4-813b-8de31279c18f for instance 54819895-e219-4341-8f1e-5543b7357315 on hv-cinder-86020 Oct 04 08:37:59.741967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-68d06b36-3f11-4fec-8bfc-b6b472364edb req-aacce309-1df0-4946-a1c0-5d2e5d7264d6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.055319 Oct 04 08:37:59.743040 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 164/328] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:37:59 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:37:59.826703 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1206723388", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7c0853df-a74a-47d5-9683-581a895763eb"}], "security_groups": [{"name": "tempest-securitygroup--1506672386"}], "key_name": "tempest-keypair-2108872820", "user_data": "}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:00.129091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:00.137807 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5afd4f02-b761-46e5-835e-5d626efd518b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.799118 Oct 04 08:38:00.138717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 165/329] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:37:59 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:00.188271 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] validate_networks() for [('7c0853df-a74a-47d5-9683-581a895763eb', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:38:00.206322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:00.207273 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:00.213522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:00.214036 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:00.214648 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:00.215359 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:00.215806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:00.216221 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:00.752092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:00.777410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:00.779342 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:00.780851 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a61ec76d-eff1-4c88-bd05-927f698eeadf tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1722 microversion: 2.20 time: 0.579234 Oct 04 08:38:00.795050 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 166/330] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:00 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1722 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:00.807005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:00.808511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:00.816639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:00.817390 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:00.817983 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:00.818908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:00.819513 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:00.820109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:00.983138 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:01.030405 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a79e53ff-cfe3-44ba-b7f7-7b4c0599330a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1270 microversion: 2.1 time: 0.233851 Oct 04 08:38:01.040519 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 167/331] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:00 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1270 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:01.168920 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:01.169774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:01.186530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:01.186969 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:01.187697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:01.189602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:01.190727 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:01.191326 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:01.831839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:01.844503 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-984560ef-d7aa-4fed-ba02-d77db1cec81a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.680716 Oct 04 08:38:01.845416 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 168/332] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:01 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:01.853641 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-9744a4e4-6474-4286-97c0-edec48721abc req-d3b990cf-3e36-403b-9e69-22278ff54256 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6dee148f-e8b5-437c-9d80-4d784091314c", "tag": "ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:01.881273 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-9744a4e4-6474-4286-97c0-edec48721abc req-d3b990cf-3e36-403b-9e69-22278ff54256 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:01.882056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-9744a4e4-6474-4286-97c0-edec48721abc req-d3b990cf-3e36-403b-9e69-22278ff54256 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:01.882666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-9744a4e4-6474-4286-97c0-edec48721abc req-d3b990cf-3e36-403b-9e69-22278ff54256 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:01.909319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-9744a4e4-6474-4286-97c0-edec48721abc req-d3b990cf-3e36-403b-9e69-22278ff54256 service nova] Creating event network-changed:ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c for instance 6dee148f-e8b5-437c-9d80-4d784091314c on hv-cinder-86020 Oct 04 08:38:01.917186 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-9744a4e4-6474-4286-97c0-edec48721abc req-d3b990cf-3e36-403b-9e69-22278ff54256 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070346 Oct 04 08:38:01.917779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 169/333] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:01.929363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:01.930055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:01.933862 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:38:01.937241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:38:01.937770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:38:01.938268 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:38:01.939235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:38:01.940916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:01.941596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:01.942020 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:01.942910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:01.943679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:01.944283 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:01.956239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:01.956730 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:01.957199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:01.957937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:01.958392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:01.958876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:02.011825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for project 5934621267af4b838889fcbfb9febaea. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:38:02.020539 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Getting quotas for user 26abb94392364bdc82f6bb508a611d44 and project 5934621267af4b838889fcbfb9febaea. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:38:02.035745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:38:02.038595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:38:02.151630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b29e8216-f27f-406a-b758-f9165e4e90c0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 2.324864 Oct 04 08:38:02.153090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 165/334] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:37:59 2022] POST /compute/v2.1/servers => generated 409 bytes in 2332 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:38:02.161971 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:02.192784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:02.201686 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:02.208586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:02.212767 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:02.213335 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:02.214082 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:02.214451 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:02.216035 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:02.217238 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d0968304-bc57-4de4-9823-c3ba3c6c4a8b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1722 microversion: 2.20 time: 0.295290 Oct 04 08:38:02.220263 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:02.222281 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:02.231167 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 170/335] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:01 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1722 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:02.243250 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9b7eaf12-76a1-44f4-b728-aea614ced172 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:02.244115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9b7eaf12-76a1-44f4-b728-aea614ced172 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:02.395675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9b7eaf12-76a1-44f4-b728-aea614ced172 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:02.415319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9b7eaf12-76a1-44f4-b728-aea614ced172 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1246 microversion: 2.1 time: 0.182597 Oct 04 08:38:02.416271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 171/336] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:02 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1246 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:02.482145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:02.491246 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-73850fff-dc18-4e23-b95a-d841f0434048 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.336715 Oct 04 08:38:02.492103 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 166/337] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:02 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:02.869844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:02.870461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:02.879425 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:02.880324 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:02.881067 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:02.881980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:02.883277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:02.883871 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:03.205717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:03.213461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2056dcec-9d9d-4571-bb46-c61426d96e91 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.347454 Oct 04 08:38:03.214044 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 172/338] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:02 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:03.247308 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:03.247850 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:03.259741 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:03.260240 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:03.260671 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:03.261404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:03.261851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:03.262272 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:03.449664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:03.450298 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:03.459411 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:03.459922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:03.460357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:03.461091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:03.461555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:03.461985 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:03.484067 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:03.495688 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:03.497836 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:03.500528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-edc287d4-21fd-4fe7-837d-b38870f8333b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1722 microversion: 2.20 time: 0.257216 Oct 04 08:38:03.502317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 167/339] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:03 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1722 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:03.523713 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:03.524927 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:03.532099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:03.532590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:03.533028 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:03.533737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:03.534189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:03.534893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:03.592777 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:03.602254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a61275ad-5741-4ee4-95d7-54346c74c69e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1250 microversion: 2.1 time: 0.156460 Oct 04 08:38:03.602885 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 173/340] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:03 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1250 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:03.895371 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:03.903895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4b5631f2-eec7-4a95-ba50-a03ecf412f97 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.390333 Oct 04 08:38:03.906590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 168/341] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:03 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:04.119205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-6078d747-8339-4935-98c3-c0b3c6ccadd5 req-b5aa3e8b-0fab-4df0-af49-ebcbc7d5c90b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2213799d-aaf1-497d-8b34-df52ba65b77a", "tag": "6b7f8247-fad9-479e-956e-2174a8b01f79"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:04.132023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-6078d747-8339-4935-98c3-c0b3c6ccadd5 req-b5aa3e8b-0fab-4df0-af49-ebcbc7d5c90b service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.132521 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-6078d747-8339-4935-98c3-c0b3c6ccadd5 req-b5aa3e8b-0fab-4df0-af49-ebcbc7d5c90b service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.132954 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-6078d747-8339-4935-98c3-c0b3c6ccadd5 req-b5aa3e8b-0fab-4df0-af49-ebcbc7d5c90b service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.146843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-6078d747-8339-4935-98c3-c0b3c6ccadd5 req-b5aa3e8b-0fab-4df0-af49-ebcbc7d5c90b service nova] Creating event network-changed:6b7f8247-fad9-479e-956e-2174a8b01f79 for instance 2213799d-aaf1-497d-8b34-df52ba65b77a on hv-cinder-86020 Oct 04 08:38:04.158226 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-6078d747-8339-4935-98c3-c0b3c6ccadd5 req-b5aa3e8b-0fab-4df0-af49-ebcbc7d5c90b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042314 Oct 04 08:38:04.158842 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 174/342] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:04.237969 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:04.238625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:04.245062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.245553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.245988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.246736 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.247196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.247614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.255055 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:04.534069 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:04.535320 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:04.545820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.546647 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.547162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.548026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.548525 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.549498 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.554211 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:04.572579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-923fcb07-b229-4569-b00e-ed2b45b7a33f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.338899 Oct 04 08:38:04.574991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 169/343] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:04 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:04.635293 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:04.635913 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:04.650918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.651880 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.652554 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.654084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.654834 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.655546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.817661 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:04.826978 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a5ce38e0-5f8e-4a7d-8bb1-334a40189404 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1298 microversion: 2.1 time: 0.200880 Oct 04 08:38:04.828552 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 170/344] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:04 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1298 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:04.933332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:04.934408 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:04.941194 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.941679 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.942111 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:04.942868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:04.943316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:04.943743 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.036881 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:05.036881 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:05.041753 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:05.043279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f60b2014-e3c8-47b7-b422-f29e29647864 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1665 microversion: 2.20 time: 0.517205 Oct 04 08:38:05.058581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 175/345] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:04 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1665 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:05.072456 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bc71647d-9cff-4147-9916-cad71db53183"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:05.074890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:05.081861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.082352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.082809 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.083509 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.083961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.084382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.147851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:05.155906 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2facddcb-cbb4-4f10-a458-f54f8caecea3 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.226514 Oct 04 08:38:05.156488 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 171/346] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:04 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:05.462708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-30106cc0-4685-43c4-bc03-72ebab5d524a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "POST /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.395788 Oct 04 08:38:05.463478 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 176/347] 10.222.0.16 () {62 vars in 1364 bytes} [Tue Oct 4 08:38:05 2022] POST /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments => generated 188 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:38:05.602636 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:05.604051 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:05.615760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.616604 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.617258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.618200 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.618893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.619531 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.625583 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:05.627258 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:05.633734 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.634571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.635224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.636153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.636850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.637474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.699707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127143) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 04 08:38:05.816157 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4192d979-faaf-4177-bd02-2f95c13baaaa tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "POST /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/action" status: 202 len: 0 microversion: 2.20 time: 0.194592 Oct 04 08:38:05.820053 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 177/348] 10.222.0.16 () {62 vars in 1319 bytes} [Tue Oct 4 08:38:05 2022] POST /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/action => generated 0 bytes in 197 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:38:05.828382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:05.829672 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:05.837775 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.838828 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.839406 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.840697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:05.841162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:05.841951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:05.852559 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:06.357589 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:06.369655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-86ef13a4-c51b-486d-89cf-d7958a9fcb36 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.770740 Oct 04 08:38:06.370414 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 172/349] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:05 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:06.377369 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:06.383657 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:06.400634 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:06.401013 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:06.401466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:06.402149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:06.404481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:06.404973 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:06.406157 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:06.422866 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:06.425579 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:06.428232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-25620f9e-5179-448e-846d-0c8c3871b516 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1751 microversion: 2.20 time: 0.605824 Oct 04 08:38:06.429683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 178/350] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:05 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1751 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:06.435603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:06.436583 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:06.456079 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:06.456613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:06.458331 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:06.460420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:06.460924 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:06.462105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:06.603489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:06.614401 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-18b0be8f-e3e0-43f6-817c-ec0c3a026d51 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1304 microversion: 2.1 time: 0.240789 Oct 04 08:38:06.615779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 173/351] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:06 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1304 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:06.761271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:06.769597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f5c2544f-c8c7-4829-8b2a-13ef5fdad67e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.338594 Oct 04 08:38:06.770649 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 179/352] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:06 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:07.393929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:07.394542 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:07.402470 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.403518 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.404269 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:07.405121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.405643 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.406133 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:07.455490 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:07.456578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:07.463829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.464675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.466036 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:07.467004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.467751 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.468408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:07.716849 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:07.729861 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-30a2f616-fa1c-45d1-aaf7-71ecb3958a1f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.339961 Oct 04 08:38:07.741746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 174/353] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:07 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:07.757314 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:07.758015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:07.764737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.765235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.765902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:07.767040 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.767508 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.767938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:07.852159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:07.867313 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:07.869793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:07.871246 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1f73dfaa-040c-4750-8239-9227f2262257 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1753 microversion: 2.20 time: 0.420592 Oct 04 08:38:07.887033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 180/354] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:07 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1753 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:07.897092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:07.897911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:07.905433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.906160 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.906826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:07.908051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:07.908690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:07.909358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:08.217255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:08.229697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:08.234631 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0cf729e2-e608-4f38-a1ae-ece4c36977e6 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.491558 Oct 04 08:38:08.236140 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 175/355] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:07 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:08.251141 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b0a0a6a8-fef4-4b65-8e25-b2984b646ebb tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.353807 Oct 04 08:38:08.251141 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 181/356] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:07 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:08.264580 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-92cfdc0a-2a68-4327-8f72-7eb39b646d6b req-ba463acd-2cc1-4d9d-8b18-129dc7249509 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c1f2b98d-4c4e-47ff-b1d4-19addcee2e27", "name": "network-vif-plugged", "status": "completed", "tag": "0e892634-2abc-4cc1-ad5b-83887511e34a"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:08.275563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-92cfdc0a-2a68-4327-8f72-7eb39b646d6b req-ba463acd-2cc1-4d9d-8b18-129dc7249509 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:08.276131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-92cfdc0a-2a68-4327-8f72-7eb39b646d6b req-ba463acd-2cc1-4d9d-8b18-129dc7249509 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:08.276794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-92cfdc0a-2a68-4327-8f72-7eb39b646d6b req-ba463acd-2cc1-4d9d-8b18-129dc7249509 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:08.297043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-92cfdc0a-2a68-4327-8f72-7eb39b646d6b req-ba463acd-2cc1-4d9d-8b18-129dc7249509 service nova] Creating event network-vif-plugged:0e892634-2abc-4cc1-ad5b-83887511e34a for instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 on hv-cinder-86020 Oct 04 08:38:08.303359 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-92cfdc0a-2a68-4327-8f72-7eb39b646d6b req-ba463acd-2cc1-4d9d-8b18-129dc7249509 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042768 Oct 04 08:38:08.303964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 182/357] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:08.559203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-c646fb03-5a23-4c58-ad71-8e52cc6fa6ab req-877cbc27-5abf-4e38-8562-3664fc864882 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5ecca0c-50c6-4218-a34e-f4ec70d2686e", "tag": "84c7f948-82c8-4e44-86f1-d8ace0e90f47"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:08.580188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-c646fb03-5a23-4c58-ad71-8e52cc6fa6ab req-877cbc27-5abf-4e38-8562-3664fc864882 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:08.581021 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-c646fb03-5a23-4c58-ad71-8e52cc6fa6ab req-877cbc27-5abf-4e38-8562-3664fc864882 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:08.581659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-c646fb03-5a23-4c58-ad71-8e52cc6fa6ab req-877cbc27-5abf-4e38-8562-3664fc864882 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:08.599860 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-c646fb03-5a23-4c58-ad71-8e52cc6fa6ab req-877cbc27-5abf-4e38-8562-3664fc864882 service nova] Creating event network-changed:84c7f948-82c8-4e44-86f1-d8ace0e90f47 for instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e on hv-cinder-86020 Oct 04 08:38:08.608413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-c646fb03-5a23-4c58-ad71-8e52cc6fa6ab req-877cbc27-5abf-4e38-8562-3664fc864882 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053798 Oct 04 08:38:08.609129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 176/358] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:08.760905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:08.761855 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:08.777755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:08.778281 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:08.778778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:08.779715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:08.780174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:08.780787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:08.906233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:08.907990 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:08.919125 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:08.919900 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:08.920339 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:08.921059 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:08.921506 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:08.922113 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:09.052901 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:09.065623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-dbcbb617-5f69-4ade-b44b-9d5d00aa82ad tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.305582 Oct 04 08:38:09.065623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 183/359] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:08 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:09.107481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:09.116850 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:09.118292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:09.119478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9ca78625-dae0-4ae8-8843-8c932e43735a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.219322 Oct 04 08:38:09.120032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 177/360] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:08 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:09.260873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:09.261504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:09.269564 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:09.270072 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:09.270509 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:09.271262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:09.271718 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:09.272146 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:09.272666 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:09.273306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:09.281122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:09.281616 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:09.282116 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:09.283199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:09.283950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:09.284377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:09.511438 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:09.527145 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-89cd2225-da12-4741-b33f-fcb6d5a4073f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.270264 Oct 04 08:38:09.528597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 184/361] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:09 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:09.549004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:09.569246 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a40624d2-d273-413d-a6fe-a008db7e63a1 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.300465 Oct 04 08:38:09.570481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 178/362] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:09 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:10.086197 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:10.086968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:10.094688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.095683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.095683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.096469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.096995 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.097419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.148126 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:10.148927 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:10.162159 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.162684 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.163119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.163820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.164271 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.164685 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.431799 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:10.440209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-404aea43-3cc9-4e3d-b5b7-c8d5c3af60b8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1267 microversion: 2.1 time: 0.357802 Oct 04 08:38:10.441083 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 185/363] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:10 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1267 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:10.447109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-8b522d42-8aee-4c65-9bcc-240f3ad9b150 req-9646bafd-738e-4b7e-af7b-e1573873bd80 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2213799d-aaf1-497d-8b34-df52ba65b77a", "name": "network-vif-plugged", "status": "completed", "tag": "6b7f8247-fad9-479e-956e-2174a8b01f79"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:10.458684 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:10.459555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-8b522d42-8aee-4c65-9bcc-240f3ad9b150 req-9646bafd-738e-4b7e-af7b-e1573873bd80 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.460048 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-8b522d42-8aee-4c65-9bcc-240f3ad9b150 req-9646bafd-738e-4b7e-af7b-e1573873bd80 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.460483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-8b522d42-8aee-4c65-9bcc-240f3ad9b150 req-9646bafd-738e-4b7e-af7b-e1573873bd80 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.467772 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:10.469211 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:10.470720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6292bb95-89bd-441b-b413-fde214a4af2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.326718 Oct 04 08:38:10.471284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 179/364] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:10 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:10.474422 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-8b522d42-8aee-4c65-9bcc-240f3ad9b150 req-9646bafd-738e-4b7e-af7b-e1573873bd80 service nova] Creating event network-vif-plugged:6b7f8247-fad9-479e-956e-2174a8b01f79 for instance 2213799d-aaf1-497d-8b34-df52ba65b77a on hv-cinder-86020 Oct 04 08:38:10.489230 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-8b522d42-8aee-4c65-9bcc-240f3ad9b150 req-9646bafd-738e-4b7e-af7b-e1573873bd80 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046875 Oct 04 08:38:10.489867 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 186/365] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:10.553487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:10.554091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:10.562175 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.562611 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.563069 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.564954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.565424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.565829 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.606712 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:10.607715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:10.624675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.625192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.626216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.627103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:10.627568 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:10.628052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:10.757148 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:10.765643 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-88ce5466-31ae-4d97-bd36-abf16a7cff32 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.215925 Oct 04 08:38:10.772464 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 180/366] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:10 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:10.819236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:10.828301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a5c4120e-3378-4f33-b405-ab33d135672a tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.225837 Oct 04 08:38:10.828301 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 187/367] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:10 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:11.464420 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:11.465090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:11.479649 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.480138 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.480573 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.481282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.481737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.482175 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.497148 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:11.497750 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:11.504957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.505440 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.505862 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.506775 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.507492 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.507906 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.709199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:11.717414 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-242a36d1-724f-4d0d-b3cc-4567dd9c5514 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1454 microversion: 2.1 time: 0.256789 Oct 04 08:38:11.718005 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 181/368] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:11 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1454 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:11.724986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:11.738644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:11.740086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:11.741411 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f4ae7822-0bef-437f-b171-0959109f4702 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.248262 Oct 04 08:38:11.742401 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:11.742578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 188/369] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:11 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:11.743096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:11.749463 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.749944 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.750377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.751864 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.752308 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.752897 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.790526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:11.791163 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:11.797449 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.798462 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.798954 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.799660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:11.800112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:11.800533 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:11.976779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:11.984735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0d543bd5-c0e5-4737-a99a-84375fb0624b tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1454 microversion: 2.1 time: 0.246219 Oct 04 08:38:11.986084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 182/370] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:11 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1454 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:11.990996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:11.991577 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:11.996519 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:12.000960 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:12.003107 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:12.003589 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:12.004105 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:12.006703 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:12.007179 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:12.007739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:12.010666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1ee15b9d-5dbd-49d4-8b28-92a1dccbd442 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.223385 Oct 04 08:38:12.012071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 189/371] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:11 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:12.031670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=12 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:12.168458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:12.178527 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-67dfbdc8-cecd-4aec-aad9-b218716914c0 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.190905 Oct 04 08:38:12.179098 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 183/372] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:11 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:12.778780 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:12.778780 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:12.781136 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=12 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:12.786503 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:12.787071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:12.787509 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:12.788280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:12.788714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:12.789205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:13.036757 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:13.036757 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:13.051869 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:13.052738 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:13.053948 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:13.054601 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:13.055883 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:13.056310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:13.229857 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:13.243366 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:13.247977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:13.250706 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2f686565-82a7-4b48-8c91-8c2981c93402 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.484959 Oct 04 08:38:13.252245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 190/373] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:12 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:13.257337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:13.258298 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:13.276667 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:13.277154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:13.277592 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:13.279059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:13.279833 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:13.280762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:13.338105 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:13.348151 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9d3fa10a-970c-489d-947b-c59086a6e797 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.316568 Oct 04 08:38:13.350239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 184/374] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:13 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:13.469157 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:13.477945 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c4d1a09f-b5e9-43f7-af52-45fb6a83ee5c tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1312 microversion: 2.1 time: 0.224352 Oct 04 08:38:13.478839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 191/375] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:13 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1312 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:14.276798 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:14.277605 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:14.285085 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:14.285565 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:14.286197 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:14.286938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:14.287556 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:14.288103 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:14.314013 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-1510d696-b037-4899-87e0-205b2ba954aa req-c755c29c-307a-450e-820b-9997e94e01c1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c1f2b98d-4c4e-47ff-b1d4-19addcee2e27", "tag": "0e892634-2abc-4cc1-ad5b-83887511e34a"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:14.326365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-1510d696-b037-4899-87e0-205b2ba954aa req-c755c29c-307a-450e-820b-9997e94e01c1 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:14.326940 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-1510d696-b037-4899-87e0-205b2ba954aa req-c755c29c-307a-450e-820b-9997e94e01c1 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:14.327425 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-1510d696-b037-4899-87e0-205b2ba954aa req-c755c29c-307a-450e-820b-9997e94e01c1 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:14.342274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-1510d696-b037-4899-87e0-205b2ba954aa req-c755c29c-307a-450e-820b-9997e94e01c1 service nova] Creating event network-changed:0e892634-2abc-4cc1-ad5b-83887511e34a for instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 on hv-cinder-86020 Oct 04 08:38:14.354960 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-1510d696-b037-4899-87e0-205b2ba954aa req-c755c29c-307a-450e-820b-9997e94e01c1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041165 Oct 04 08:38:14.356022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 192/376] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:38:14.378925 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:14.381233 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:14.389107 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:14.389622 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:14.390175 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:14.391797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:14.392286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:14.392719 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:14.536165 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:14.546845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:14.549729 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:14.552701 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3b7f1b08-af40-4e4f-9178-4384dd96042e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.278393 Oct 04 08:38:14.552701 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 185/377] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:14 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:14.557940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:14.558739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:14.566057 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:14.567031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:14.567698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:14.568635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:14.569289 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:14.569904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:14.586677 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:14.625291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:14.638125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-71514fa3-cd6a-419b-a7e3-411ebe0d1b3b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.269059 Oct 04 08:38:14.639046 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 193/378] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:14 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:14.731923 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:14.741388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b7812da8-9e93-4e9c-ba07-cce77ff19d85 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1494 microversion: 2.1 time: 0.187313 Oct 04 08:38:14.741960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 186/379] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:14 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1494 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:15.585773 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:15.586801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:15.593309 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:15.593921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:15.606539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:15.611122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:15.612060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:15.613305 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:15.615285 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:15.616991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:15.619539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:15.621344 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:15.621879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:15.623111 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:15.623409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:15.627947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:15.725606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:38:15.970906 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9ece2419-133e-47a4-a6dc-5ff9af3ca25e tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 204 len: 0 microversion: 2.1 time: 0.388727 Oct 04 08:38:15.979023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 187/380] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:38:15 2022] DELETE /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:15.996921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:16.004306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:16.012054 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:16.026454 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:16.028602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:16.030059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-64293e71-bd2b-4dc1-9fd6-2e764607174f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.440604 Oct 04 08:38:16.034111 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:16.037877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:16.040404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:16.043620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 194/381] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:15 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:16.045241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:16.050830 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:16.054946 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:16.056817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:16.057714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:16.064568 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=2,nova_api:UPDATE=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.066464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:16.067192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:16.067962 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:16.068879 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:16.069647 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:16.070260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:16.447540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:16.462258 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-42dc3764-f31b-4c84-bf75-5838f2647af5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.417346 Oct 04 08:38:16.466125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 195/382] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:16 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:16.471530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-5be51d87-5cb4-4189-b250-14ee874c7b89 req-58a0eb75-b94c-4150-8bf4-974c737a04b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5ecca0c-50c6-4218-a34e-f4ec70d2686e", "name": "network-vif-plugged", "status": "completed", "tag": "84c7f948-82c8-4e44-86f1-d8ace0e90f47"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:16.486731 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-5be51d87-5cb4-4189-b250-14ee874c7b89 req-58a0eb75-b94c-4150-8bf4-974c737a04b6 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:16.487220 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-5be51d87-5cb4-4189-b250-14ee874c7b89 req-58a0eb75-b94c-4150-8bf4-974c737a04b6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:16.487687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-5be51d87-5cb4-4189-b250-14ee874c7b89 req-58a0eb75-b94c-4150-8bf4-974c737a04b6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:16.488974 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:16.501887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-5be51d87-5cb4-4189-b250-14ee874c7b89 req-58a0eb75-b94c-4150-8bf4-974c737a04b6 service nova] Creating event network-vif-plugged:84c7f948-82c8-4e44-86f1-d8ace0e90f47 for instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e on hv-cinder-86020 Oct 04 08:38:16.508428 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-5be51d87-5cb4-4189-b250-14ee874c7b89 req-58a0eb75-b94c-4150-8bf4-974c737a04b6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040537 Oct 04 08:38:16.509817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 196/383] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:16.518041 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-100d537b-5b03-449f-8d47-8b47f862508d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.536396 Oct 04 08:38:16.519320 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 188/384] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:15 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:17.061900 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:17.062498 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:17.070139 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:17.070646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:17.071078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:17.071755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:17.072208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:17.072622 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:17.087616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.215729 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:17.225793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:17.227566 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:17.228880 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0c20fc15-d152-4228-8905-7f147ab52df4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.171647 Oct 04 08:38:17.229445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 197/385] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:17 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:17.486255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:17.486893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:17.489393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.494634 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:17.495119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:17.495552 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:17.496248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:17.496698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:17.497113 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:17.541267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:17.542197 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:17.549294 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:17.550106 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:17.550106 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:17.551101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:17.551255 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:17.551683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:17.721865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:17.730168 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e4de4456-7506-45be-96ae-b4a5b8fbba42 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.248645 Oct 04 08:38:17.730902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 189/386] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:17 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:17.758291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:17.767068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5ebfb73f-f373-492d-803b-07bef7aab402 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.228887 Oct 04 08:38:17.767737 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 198/387] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:17 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:18.260688 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:18.261708 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:18.271969 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:18.272461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:18.272902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:18.273626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:18.274084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:18.274513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:18.434764 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:18.450283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:18.453342 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:18.456169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-168a91f8-6aef-44b6-a5b8-af3e6856bdff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.202596 Oct 04 08:38:18.458626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 190/388] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:18 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:18.754628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:18.755378 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:18.762510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:18.763086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:18.763568 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:18.764309 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:18.764800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:18.765743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:18.795496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:18.797145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:18.804945 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:18.805806 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:18.806239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:18.807036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:18.807639 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:18.808214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:18.985378 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:18.993713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c940c42a-31a8-40d8-b739-fd9ca4bab1e7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.243681 Oct 04 08:38:18.994744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 199/389] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:18 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:19.015820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:19.023878 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d0ee8fb7-6cac-4781-8575-596a27e9495d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.237339 Oct 04 08:38:19.024642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 191/390] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:18 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:19.483825 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:19.484438 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:19.491455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:19.492068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:19.492604 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:19.493545 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:19.494092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:19.494639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:19.677345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:19.686668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:19.688203 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:19.689327 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-82ca14a6-1beb-495d-a38f-d6b4c9b5861e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.209628 Oct 04 08:38:19.689888 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 200/391] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:19 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:20.017941 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:20.018572 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:20.025450 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:20.025950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:20.026391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:20.027125 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:20.027584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:20.028010 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:20.051042 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:20.051855 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:20.058394 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:20.058826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:20.059501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:20.060220 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:20.060861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:20.061344 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:20.248592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:20.262095 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1babb5c3-4fda-4ad7-bcdf-35dd70dbce15 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.248485 Oct 04 08:38:20.263106 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 192/392] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:20 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:20.285449 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:20.293806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-716d2c23-01b1-4a0e-9bea-3f0f2a74ea0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.249972 Oct 04 08:38:20.294483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 201/393] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:20 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:20.716391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:20.716882 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:20.724166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:20.724655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:20.725314 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:20.726027 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:20.726474 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:20.726918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:20.823290 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1374518852", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-391951627", "security_groups": [{"name": "tempest-secgroup-smoke-589160591"}], "networks": [{"uuid": "5a3147e2-6e01-4190-a593-7d5113358d23"}], "block_device_mapping_v2": [{"uuid": "109e36b4-d442-464d-a16e-e3d04ccde83a", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:20.911222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:20.921855 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:20.924560 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:20.926005 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6ea850fc-d5ca-4177-aeaa-eae2c0aacbba tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.213674 Oct 04 08:38:20.937499 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 193/394] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:20 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:20.973596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] validate_networks() for [('5a3147e2-6e01-4190-a593-7d5113358d23', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:38:21.288983 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:21.290267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:21.297457 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:21.299725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:21.300373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:21.301257 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:21.301713 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:21.302135 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:21.443718 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:21.452541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-da85fc26-4573-4a47-9401-ecaf2b9fc1a7 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.167869 Oct 04 08:38:21.453171 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 194/395] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:21 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:21.458639 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:21.459266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:21.465616 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:21.466158 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:21.466626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:21.467348 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:21.467803 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:21.468228 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:21.613346 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:21.621298 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-245a068d-ebd8-4bf7-8489-08e34820b340 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.166556 Oct 04 08:38:21.621880 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 195/396] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:21 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:21.951571 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:21.952188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:21.959952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:21.960446 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:21.960963 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:21.961591 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:21.962101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:21.962488 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:22.099733 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:22.113790 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:22.115849 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:22.116760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0d8150bd-45d3-48f2-b065-bb10528f3df1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.169193 Oct 04 08:38:22.117341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 196/397] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:21 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:22.478198 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:22.478198 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:22.484616 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:22.485101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:22.486149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:22.486348 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:22.487133 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:22.487295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:22.634254 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:22.643437 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4586e0c5-31f9-4677-9a5e-80601ce66fc8 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.169932 Oct 04 08:38:22.644031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 197/398] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:22 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:22.649456 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:22.650190 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:22.658751 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:22.659252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:22.660309 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:22.660769 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:22.661224 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:22.663234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:22.833088 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:22.842454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-eb98057b-283b-4658-9d73-a5e0e89e1aa3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.196808 Oct 04 08:38:22.842682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 198/399] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:22 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:23.142996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:23.143738 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:23.161461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:23.162058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:23.162581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:23.163355 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:23.163884 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:23.164370 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:23.325827 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:23.336319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:23.337789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:23.339017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9143e29c-dbcd-4bc4-a9d9-ea660b9bdcd1 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.200984 Oct 04 08:38:23.352754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 199/400] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:23 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:23.669051 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:23.669726 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:23.677535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:23.678360 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:23.679141 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:23.680154 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:23.680644 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:23.681341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:23.831746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:23.840073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2d519227-059c-4e77-b72c-1418a7730df5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.175241 Oct 04 08:38:23.850880 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 200/401] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:23 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:23.867608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:23.868228 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:23.875852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:23.876357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:23.876800 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:23.887222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:23.887727 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:23.888177 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:23.908688 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:24.038458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:24.048117 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6d581a31-cef1-4845-8e77-b9616c92c78f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.184524 Oct 04 08:38:24.048695 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 201/402] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:23 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:24.367995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:24.368155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:24.374799 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:24.375345 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:24.376306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:24.377022 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:24.377505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:24.377934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:24.542526 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:24.552856 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:24.554319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:24.555536 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-01266c94-2337-445c-a524-8b6186a076dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.192371 Oct 04 08:38:24.556098 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 202/403] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:24 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:24.873277 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:24.873945 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:24.881624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:24.882106 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:24.882536 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:24.883258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:24.883704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:24.884121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:25.044071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:25.053016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-aa1d630f-6ec5-4b8d-9fdc-f2925cbdf8ab tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.183520 Oct 04 08:38:25.053616 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 203/404] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:24 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:25.071835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:25.072538 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:25.079689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:25.080523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:25.081318 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:25.082356 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:25.082906 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:25.084166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:25.245362 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:25.254816 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5337cb55-b079-456d-bfcf-40bb4c7bdcfa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.186714 Oct 04 08:38:25.255391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 204/405] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:25 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:25.581575 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:25.582192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:25.593706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:25.593706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:25.593706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:25.593706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:25.594908 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:25.594908 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:25.744943 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:25.754251 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:25.755736 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:25.756942 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-50c562fd-1d33-43c6-955f-df9264297060 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.179696 Oct 04 08:38:25.757760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 205/406] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:25 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:26.076511 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:26.077216 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:26.084585 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:26.085066 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:26.085502 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:26.086208 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:26.086691 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:26.087113 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:26.224303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:26.232255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ffd7da78-0975-406e-96a3-c76d15cb2de2 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.159826 Oct 04 08:38:26.232826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 206/407] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:26 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:26.277357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:26.278107 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:26.284470 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:26.284943 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:26.285366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:26.286054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:26.286493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:26.286932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:26.425081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:26.432730 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c369ea87-fe7e-4322-adb2-408fab8e9e18 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.159303 Oct 04 08:38:26.433305 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 207/408] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:26 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:26.784681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:26.786214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:26.793324 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:26.793835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:26.794343 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:26.795382 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:26.795802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:26.796373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:26.976373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:26.986031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:26.987614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:26.988781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-fc5aaf7f-8c4e-4861-bfd8-f8f3dda516f5 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.209457 Oct 04 08:38:26.989346 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 208/409] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:26 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:27.260887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:27.261579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:27.268930 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:27.269414 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:27.269846 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:27.270542 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:27.271007 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:27.271424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:27.405193 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:27.413334 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e5accfef-dced-4d05-83f5-a4fc1f682238 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.156229 Oct 04 08:38:27.413842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 209/410] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:27 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:27.455619 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:27.456225 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:27.462714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:27.463190 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:27.463625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:27.464300 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:27.464741 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:27.465156 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:27.992231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:28.000153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2916be8f-02d1-4b6e-bcc2-c5fd9e57a105 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.548396 Oct 04 08:38:28.001342 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 210/411] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:27 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:28.014752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:28.015355 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:28.021765 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:28.022243 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:28.022708 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:28.023397 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:28.023847 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:28.024271 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:28.188075 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:28.198064 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:28.199601 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:28.200926 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4f25d39a-a720-4d89-acd7-57a14a32f762 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.190051 Oct 04 08:38:28.201481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 211/412] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:28 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:28.436699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:28.437420 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:28.444358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:28.444835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:28.445261 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:28.445945 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:28.446381 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:28.446817 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:28.661202 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:28.669227 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-63a7ff49-cfb9-475c-848e-7d7c99dc06cd tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1500 microversion: 2.1 time: 0.236282 Oct 04 08:38:28.669814 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 212/413] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:28 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1500 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:28.790158 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-2f3e7ba9-902d-4811-901f-5afac0f37a96 req-01f61955-2967-46e3-b022-11ba6b5be2b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2213799d-aaf1-497d-8b34-df52ba65b77a", "name": "network-vif-deleted", "tag": "6b7f8247-fad9-479e-956e-2174a8b01f79"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:28.800765 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2f3e7ba9-902d-4811-901f-5afac0f37a96 req-01f61955-2967-46e3-b022-11ba6b5be2b0 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:28.801234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2f3e7ba9-902d-4811-901f-5afac0f37a96 req-01f61955-2967-46e3-b022-11ba6b5be2b0 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:28.801658 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2f3e7ba9-902d-4811-901f-5afac0f37a96 req-01f61955-2967-46e3-b022-11ba6b5be2b0 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:28.813750 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-2f3e7ba9-902d-4811-901f-5afac0f37a96 req-01f61955-2967-46e3-b022-11ba6b5be2b0 service nova] Creating event network-vif-deleted:6b7f8247-fad9-479e-956e-2174a8b01f79 for instance 2213799d-aaf1-497d-8b34-df52ba65b77a on hv-cinder-86020 Oct 04 08:38:28.818995 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-2f3e7ba9-902d-4811-901f-5afac0f37a96 req-01f61955-2967-46e3-b022-11ba6b5be2b0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032442 Oct 04 08:38:28.819528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 213/414] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:29.024568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:29.025262 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:29.032284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:29.032524 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:29.032950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:29.033815 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:29.034260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:29.034717 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:29.182447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:29.190855 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e995324d-2aee-47e7-944c-535039a68ae7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1372 microversion: 2.1 time: 0.170360 Oct 04 08:38:29.191435 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 214/415] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:29 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1372 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:29.229005 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:29.230609 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:29.239779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:29.240342 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:29.240805 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:29.241560 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:29.242077 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:29.242589 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:29.395867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:29.406768 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:29.408299 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:29.409540 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d3c6c70d-86ab-4fe7-82ad-3b24f20e7184 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.186881 Oct 04 08:38:29.410119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 215/416] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:29 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:29.693263 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:29.693957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:29.700491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:29.700980 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:29.701421 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:29.702310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:29.702913 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:29.703435 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:29.743250 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:29.743745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:29.744415 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:29.744877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lazy-loading 'fault' on Instance uuid 2213799d-aaf1-497d-8b34-df52ba65b77a {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:38:29.788283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:29.796301 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9fb43ca6-e3ff-4dab-87de-f4e2ce38e1f5 tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1302 microversion: 2.1 time: 0.106891 Oct 04 08:38:29.796892 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 216/417] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:29 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1302 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:30.214816 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:30.215468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:30.222460 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.222991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.223423 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.224113 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.224556 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.224973 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.290371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=27 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:30.372050 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:30.381617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ddcc73ba-d160-4e8e-aa71-d2bbe1c4a1b0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1565 microversion: 2.1 time: 0.170539 Oct 04 08:38:30.382224 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 217/418] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:30 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1565 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:30.404810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:30.405318 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:30.412377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.412797 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.413214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.413939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.414390 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.414856 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.430782 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:30.497489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-824cd5fa-d952-44fe-9ceb-7c31e1f9fdee tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096762 Oct 04 08:38:30.498079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 218/419] 10.222.0.16 () {58 vars in 1275 bytes} [Tue Oct 4 08:38:30 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:38:30.504281 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:30.504885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:30.511295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.511770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.512195 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.512880 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.513320 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.513739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.673822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:30.701960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:30.709960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:30.711344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2040ed94-78aa-4c15-8034-13d98b1dae14 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.211060 Oct 04 08:38:30.711959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 219/420] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:30 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:30.822989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:30.823617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:30.830886 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.832406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.832840 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.833816 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.834344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.834812 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.841640 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=14 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:30.883300 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:30.883960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:30.884523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:30.885231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lazy-loading 'fault' on Instance uuid 2213799d-aaf1-497d-8b34-df52ba65b77a {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:38:30.936871 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '76b2078f1f64440e9d8faca3bcf98d52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f758e7615232486a93a915760633b702', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:30.944999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7bcf3b3c-b3f8-4cf5-87e2-74572e9ba46d tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 200 len: 1302 microversion: 2.1 time: 0.126705 Oct 04 08:38:30.945634 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 220/421] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:30 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 1302 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:31.748830 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:31.749714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:31.756541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:31.757025 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:31.757458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:31.758155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:31.758627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:31.759049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:31.908947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:31.917728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:31.919213 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:31.920388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-15b6d91b-b58c-4478-934b-3681e2384dc8 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.175447 Oct 04 08:38:31.920957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 221/422] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:31 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:31.931455 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-81832ee9-c3dd-4282-aefc-f9516c853a3c req-17a85fa6-649e-404d-8025-6c7009ecb3cf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5ecca0c-50c6-4218-a34e-f4ec70d2686e", "tag": "84c7f948-82c8-4e44-86f1-d8ace0e90f47"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:31.942720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-81832ee9-c3dd-4282-aefc-f9516c853a3c req-17a85fa6-649e-404d-8025-6c7009ecb3cf service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:31.943307 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-81832ee9-c3dd-4282-aefc-f9516c853a3c req-17a85fa6-649e-404d-8025-6c7009ecb3cf service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:31.943912 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-81832ee9-c3dd-4282-aefc-f9516c853a3c req-17a85fa6-649e-404d-8025-6c7009ecb3cf service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:31.959677 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-81832ee9-c3dd-4282-aefc-f9516c853a3c req-17a85fa6-649e-404d-8025-6c7009ecb3cf service nova] Creating event network-changed:84c7f948-82c8-4e44-86f1-d8ace0e90f47 for instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e on hv-cinder-86020 Oct 04 08:38:31.964669 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-81832ee9-c3dd-4282-aefc-f9516c853a3c req-17a85fa6-649e-404d-8025-6c7009ecb3cf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042583 Oct 04 08:38:31.965422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 222/423] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:31.974322 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:31.975062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] [instance: 2213799d-aaf1-497d-8b34-df52ba65b77a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:31.982160 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:31.982693 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:31.983130 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:31.983851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:31.984300 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:31.985608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:32.010260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] HTTP exception thrown: Instance 2213799d-aaf1-497d-8b34-df52ba65b77a could not be found. Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Request method failure captured: Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a HTTP/1.1 Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_C4Ow5MXLpWaNKHXTM5mvowxqPB7vq1v1xGYxH5rzF3OukQlkrenYbx2TbY7ji0A3328RRzx67kO6Ym263GPs0VGBWhhKTT8WXtlpdgiv3HDkSIwRSu08NMfFPWnqg-u9PQ799I2hoBYABbnECkvOijbfVVtjLXlg2nK7CmFtKIp44 Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: f758e7615232486a93a915760633b702 Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-VolumesActionsTest-1354781618 Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:38:32.013746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f758e7615232486a93a915760633b702"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f758e7615232486a93a915760633b702"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f758e7615232486a93a915760633b702"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f758e7615232486a93a915760633b702", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-VolumesActionsTest-1354781618 Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: f758e7615232486a93a915760633b702 Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-VolumesActionsTest-1354781618 Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-VolumesActionsTest-1354781618-project Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 76b2078f1f64440e9d8faca3bcf98d52 Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-VolumesActionsTest-1354781618-project Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 2213799d-aaf1-497d-8b34-df52ba65b77a could not be found. Oct 04 08:38:32.015189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': '2213799d-aaf1-497d-8b34-df52ba65b77a'} Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2213799d-aaf1-497d-8b34-df52ba65b77a could not be found. Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:38:32.016388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2213799d-aaf1-497d-8b34-df52ba65b77a could not be found. Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2213799d-aaf1-497d-8b34-df52ba65b77a could not be found. Oct 04 08:38:32.017624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:32.019909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] Returning 404 to user: Instance 2213799d-aaf1-497d-8b34-df52ba65b77a could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:38:32.019909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-870ac8ee-efbc-4440-b223-ed07b7b7aa3b tempest-VolumesActionsTest-1354781618 tempest-VolumesActionsTest-1354781618-project] 10.222.0.16 "GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a" status: 404 len: 111 microversion: 2.1 time: 0.046599 Oct 04 08:38:32.019909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 223/424] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:31 2022] GET /compute/v2.1/servers/2213799d-aaf1-497d-8b34-df52ba65b77a => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:38:32.949015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:32.949650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:32.957534 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:32.958028 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:32.958463 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:32.959244 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:32.959687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:32.960103 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:33.112735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:33.121422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:33.122887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:33.124334 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a2aeba49-e674-4781-94d6-90cfd9d34d6b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.180640 Oct 04 08:38:33.124925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 224/425] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:32 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:34.152282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:34.152921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:34.164341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:34.164828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:34.165260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:34.165962 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:34.166406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:34.166873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:34.312385 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:34.322986 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:34.324697 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:34.325907 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cbd14adc-26b5-49d5-a2cf-65a87937a004 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.177607 Oct 04 08:38:34.326473 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 225/426] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:34 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:35.359158 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:35.359825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:35.366223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:35.366734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:35.367161 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:35.367844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:35.368283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:35.368696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:35.511384 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:35.524388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:35.525868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:35.527073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-271c2c33-95c5-48d3-a1e7-da9119cc13cd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.171825 Oct 04 08:38:35.527633 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 226/427] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:35 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:36.553647 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:36.554323 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:36.560800 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:36.561273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:36.561704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:36.562424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:36.563560 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:36.564002 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:36.703331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:36.712250 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:36.713704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:36.714913 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d30191c6-c6e6-472a-beb8-413a8f74a718 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.165191 Oct 04 08:38:36.715485 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 227/428] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:36 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:36.934024 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:36.934512 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:36.940859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:36.941341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:36.941775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:36.942582 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:36.943059 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:36.943480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:36.967972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d7f5b0c0-9b40-490f-9ede-468171dfec77 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/ips" status: 200 len: 146 microversion: 2.1 time: 0.037759 Oct 04 08:38:36.968555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 228/429] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:38:36 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/ips => generated 146 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:38:37.754581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:37.760593 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:37.790140 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:37.790920 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:37.791601 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:37.792666 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:37.793445 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:37.794434 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:38.049454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:38.066172 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:38.067799 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:38.069240 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-04e41ce8-bf36-4926-804b-11cf0c80f025 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.330761 Oct 04 08:38:38.082194 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 229/430] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:37 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:39.107523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:39.110583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:39.119564 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:39.120015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:39.120794 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:39.121946 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:39.122853 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:39.123487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:39.298917 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:39.308327 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:39.309795 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:39.311046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-451b2850-d06d-496b-97dc-baa3ad950a87 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.209032 Oct 04 08:38:39.329829 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 230/431] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:39 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:38:40.139587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-f7470f97-5e95-4358-b8ce-12cbad9e1f85 req-3bc59eb3-a6f0-42bf-b2e6-54532163b626 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54819895-e219-4341-8f1e-5543b7357315", "name": "network-vif-plugged", "status": "completed", "tag": "159277a9-e8de-47c4-813b-8de31279c18f"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:40.154366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f7470f97-5e95-4358-b8ce-12cbad9e1f85 req-3bc59eb3-a6f0-42bf-b2e6-54532163b626 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:40.155321 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f7470f97-5e95-4358-b8ce-12cbad9e1f85 req-3bc59eb3-a6f0-42bf-b2e6-54532163b626 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:40.156018 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f7470f97-5e95-4358-b8ce-12cbad9e1f85 req-3bc59eb3-a6f0-42bf-b2e6-54532163b626 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:40.171415 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-f7470f97-5e95-4358-b8ce-12cbad9e1f85 req-3bc59eb3-a6f0-42bf-b2e6-54532163b626 service nova] Creating event network-vif-plugged:159277a9-e8de-47c4-813b-8de31279c18f for instance 54819895-e219-4341-8f1e-5543b7357315 on hv-cinder-86020 Oct 04 08:38:40.180325 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-f7470f97-5e95-4358-b8ce-12cbad9e1f85 req-3bc59eb3-a6f0-42bf-b2e6-54532163b626 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043584 Oct 04 08:38:40.181675 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 231/432] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:38:40.339074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:40.340070 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:40.350948 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:40.351919 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:40.352463 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:40.353509 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:40.354127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:40.354746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:40.615084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:40.628573 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:40.630383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:40.631844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f9aa8004-9652-4b2b-a79b-f237c94779b2 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.297054 Oct 04 08:38:40.632748 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 232/433] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:40 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:41.661450 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:41.662388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:41.673053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:41.673972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:41.674965 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:41.676119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:41.676727 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:41.677314 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:41.886707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:41.897582 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:41.899578 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:41.900839 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7a8c61a5-9289-4fe6-afd8-db1a40a2db6d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.244177 Oct 04 08:38:41.902044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 233/434] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:41 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:41.907875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4418b356-105a-4f4d-a378-da2da99c3d91"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:41.910468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:41.918353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:41.919529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:41.920265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:41.921315 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:41.922323 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:41.923023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:42.363526 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-da9b9391-ec80-4021-8730-370dfdb02b63 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.459446 Oct 04 08:38:42.364716 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 234/435] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:38:41 2022] POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 194 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:42.945353 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:42.946877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:42.957100 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:42.958378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:42.958378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:42.958879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:42.959818 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:42.960668 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:43.170233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:43.180918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:43.182532 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:43.183973 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1b283737-4ff5-49b0-a874-fd5ed6e3f4c3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1809 microversion: 2.20 time: 0.243879 Oct 04 08:38:43.185074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 235/436] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:42 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1809 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:44.215517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:44.217069 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:44.225799 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:44.226281 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:44.226883 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:44.228449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:44.229528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:44.231016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:44.238029 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.410046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:44.424060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:44.427015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:44.429668 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-801630db-4108-4059-8706-340b9bbe29c0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1796 microversion: 2.20 time: 0.220019 Oct 04 08:38:44.431036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 236/437] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:38:44 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1796 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:38:50.116391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-569286ee-f8df-464d-a798-a728bbce26f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:50.119378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:50.125546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-569286ee-f8df-464d-a798-a728bbce26f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:50.126162 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-569286ee-f8df-464d-a798-a728bbce26f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:50.126644 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-569286ee-f8df-464d-a798-a728bbce26f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:50.195934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-569286ee-f8df-464d-a798-a728bbce26f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Going to try to stop instance {{(pid=127142) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 04 08:38:50.255848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-569286ee-f8df-464d-a798-a728bbce26f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action" status: 202 len: 0 microversion: 2.1 time: 0.142944 Oct 04 08:38:50.256655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 237/438] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:38:50 2022] POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action => generated 0 bytes in 145 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:38:50.265957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:50.266706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:50.275826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:50.276331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:50.277163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:50.277500 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:50.277966 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:50.278394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:50.547701 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:50.557357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cf1143ca-2736-4fc6-a5ed-19f06bc159f2 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.296276 Oct 04 08:38:50.571449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 238/439] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:50 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:51.584758 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:51.586094 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:51.596531 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:51.597051 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:51.597559 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:51.598271 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:51.598762 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:51.599194 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:51.772674 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:51.783595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5cc30aa1-3e7b-4ed8-ad48-53af7ab2abbe tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.204565 Oct 04 08:38:51.784419 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 239/440] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:51 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:52.813851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:52.814914 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:52.833424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:52.834641 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:52.835675 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:52.837366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:52.839338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:52.839338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:53.073659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:53.091149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2ee89935-870b-4980-b774-912962d18865 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.285203 Oct 04 08:38:53.098835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 240/441] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:52 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:54.133826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:54.134879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:54.143333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.143923 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.144448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.145252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.145771 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.146263 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.362803 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:54.382755 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-41e38fab-c816-42c8-8b03-8f88f7178b28 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.250721 Oct 04 08:38:54.382755 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 241/442] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:54 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:54.388672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:54.389384 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:54.400963 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.401752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.402464 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.403811 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.404472 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.405129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.435365 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-61a306c6-925e-44b5-8bd0-a2fc6e65243a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c/ips" status: 200 len: 150 microversion: 2.1 time: 0.053257 Oct 04 08:38:54.446099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 242/443] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:38:54 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c/ips => generated 150 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:54.451824 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7fc3975c-c233-43c0-aa63-cdcf158a4760"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:54.455625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:54.462628 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.463114 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.463557 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.464303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.464751 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.465197 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.568695 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Invalid volume: volume 7fc3975c-c233-43c0-aa63-cdcf158a4760 is already attached to instances: a8351183-2123-48d0-9f68-c06ab2829d71 Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: POST /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c/os-volume_attachments HTTP/1.1 Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 74 Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:38:54.570753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {"volumeAttachment": {"volumeId": "7fc3975c-c233-43c0-aa63-cdcf158a4760"}} Oct 04 08:38:54.573239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Invalid volume: volume 7fc3975c-c233-43c0-aa63-cdcf158a4760 is already attached to instances: a8351183-2123-48d0-9f68-c06ab2829d71 Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'server_id': '6dee148f-e8b5-437c-9d80-4d784091314c', 'body': {'volumeAttachment': {'volumeId': '7fc3975c-c233-43c0-aa63-cdcf158a4760'}}} Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 7fc3975c-c233-43c0-aa63-cdcf158a4760 is already attached to instances: a8351183-2123-48d0-9f68-c06ab2829d71 Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:54.574489 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 7fc3975c-c233-43c0-aa63-cdcf158a4760 is already attached to instances: a8351183-2123-48d0-9f68-c06ab2829d71 Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:38:54.576132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 7fc3975c-c233-43c0-aa63-cdcf158a4760 is already attached to instances: a8351183-2123-48d0-9f68-c06ab2829d71 Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 400 to user: Invalid volume: volume 7fc3975c-c233-43c0-aa63-cdcf158a4760 is already attached to instances: a8351183-2123-48d0-9f68-c06ab2829d71 {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2a9441e4-a2ec-4bf1-a3a6-88c40c89d5f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.125492 Oct 04 08:38:54.577471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 243/444] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:38:54 2022] POST /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c/os-volume_attachments => generated 174 bytes in 127 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:38:54.588377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:54.589281 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:54.602060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.602988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.603789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.605076 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.605916 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.606737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.675802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:38:54.841058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cfd5e777-7d3a-4f7a-bb27-8482fe28e0eb tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 204 len: 0 microversion: 2.1 time: 0.259148 Oct 04 08:38:54.842611 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 244/445] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:38:54 2022] DELETE /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:38:54.856952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:54.858469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:54.870885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.871722 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.872460 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:54.873661 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:54.875269 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:54.875985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:55.082023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:55.091973 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e5c08ecd-1f05-4723-8bec-bf3e94986789 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1681 microversion: 2.1 time: 0.243536 Oct 04 08:38:55.092985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 245/446] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:54 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1681 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:55.407215 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:55.408456 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:55.418633 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:55.419169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:55.419635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:55.420427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:55.420938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:55.421441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:55.597865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:55.609700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-99455f1a-bce1-4dbc-ae0f-f083131f4c08 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.207390 Oct 04 08:38:55.610294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 246/447] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:55 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:56.119947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:56.120563 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:56.129556 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:56.130176 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:56.130886 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:56.131590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:56.132261 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:56.132744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:56.308466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:56.319858 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3f640a89-0175-4c52-bd41-cc1fc6e85ffe tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1681 microversion: 2.1 time: 0.203712 Oct 04 08:38:56.320786 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 247/448] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:56 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1681 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:56.636009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:56.636891 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:56.646888 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:56.647529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:56.648134 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:56.649140 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:56.649800 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:56.650408 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:56.911656 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:56.927358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4a01bcf1-110f-4d56-abdb-91a1d9d5fad0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.297017 Oct 04 08:38:56.928339 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 248/449] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:56 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:57.347524 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:57.348393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:57.355745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:57.356831 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:57.357336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:57.358186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:57.359078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:57.359750 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:57.611419 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:57.622440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2a923675-57d9-41e5-af91-c06db23586f3 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1681 microversion: 2.1 time: 0.281002 Oct 04 08:38:57.627074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 249/450] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:57 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1681 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:57.971393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:57.973738 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:57.987323 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:57.988777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:57.989872 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:57.992104 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:57.993885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:57.995070 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:58.031588 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:58.442487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:58.464350 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e93c1f9e-8058-4b71-a4d2-a2f8cc1e3b40 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.501306 Oct 04 08:38:58.480263 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 250/451] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:57 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:38:58.488868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-f6511cfe-f032-4ab4-90b1-3575f31ebe01 req-e7b85582-27ae-4476-822b-2511baa7dd33 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6dee148f-e8b5-437c-9d80-4d784091314c", "name": "network-vif-deleted", "tag": "ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:38:58.500648 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f6511cfe-f032-4ab4-90b1-3575f31ebe01 req-e7b85582-27ae-4476-822b-2511baa7dd33 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:58.501813 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f6511cfe-f032-4ab4-90b1-3575f31ebe01 req-e7b85582-27ae-4476-822b-2511baa7dd33 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:58.502265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-f6511cfe-f032-4ab4-90b1-3575f31ebe01 req-e7b85582-27ae-4476-822b-2511baa7dd33 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:58.517372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-f6511cfe-f032-4ab4-90b1-3575f31ebe01 req-e7b85582-27ae-4476-822b-2511baa7dd33 service nova] Creating event network-vif-deleted:ccd88958-a0b5-41e2-b7af-10bf8c1fbe1c for instance 6dee148f-e8b5-437c-9d80-4d784091314c on hv-cinder-86020 Oct 04 08:38:58.523315 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-f6511cfe-f032-4ab4-90b1-3575f31ebe01 req-e7b85582-27ae-4476-822b-2511baa7dd33 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041425 Oct 04 08:38:58.524286 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 251/452] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:38:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:38:58.673856 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:58.674679 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:58.685156 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:58.685687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:58.686239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:58.687980 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:58.688321 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:58.688778 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:58.742971 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:58.743383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:58.744050 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:58.744716 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 6dee148f-e8b5-437c-9d80-4d784091314c {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:38:59.104313 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:59.121187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-628155f6-3765-436a-8cf6-6422db5dcb0b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 200 len: 1331 microversion: 2.1 time: 0.461071 Oct 04 08:38:59.125171 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 252/453] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:58 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 1331 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:38:59.493800 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:38:59.494431 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:38:59.502113 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:59.502675 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:59.503124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:59.503855 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:38:59.504490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:38:59.505264 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:38:59.739349 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:38:59.753666 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2b26488b-52f7-4783-a18e-ceeca59fbd3b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.263358 Oct 04 08:38:59.755359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 253/454] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:38:59 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:00.174503 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:00.175548 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:00.181970 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:00.182783 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:00.183496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:00.184456 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:00.184894 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:00.185491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:00.203864 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c HTTP/1.1 Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:39:00.205517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:39:00.206810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': '6dee148f-e8b5-437c-9d80-4d784091314c'} Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:39:00.207513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:39:00.208985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:00.210367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:39:00.210367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-72a898a7-2657-413d-992f-c394b4f9706e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 404 len: 111 microversion: 2.1 time: 0.037972 Oct 04 08:39:00.210735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 254/455] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:00 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:39:00.215699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f88a8393-3fd7-41c2-945b-2fa71fc3c89e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '.version_select at 0x7f30a0a21a60>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:00.239513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f88a8393-3fd7-41c2-945b-2fa71fc3c89e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-355187951" status: 202 len: 0 microversion: 2.1 time: 0.027623 Oct 04 08:39:00.240214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 255/456] 10.222.0.16 () {58 vars in 1218 bytes} [Tue Oct 4 08:39:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-355187951 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:39:00.779969 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:00.781492 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:00.789331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:00.790381 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:00.790997 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:00.791728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:00.792193 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:00.792623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:01.034454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:01.045016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-33fd3f71-5bb5-42e3-95d2-cc229b843803 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.270383 Oct 04 08:39:01.047087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 256/457] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:00 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:01.096867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:01.097363 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:01.104009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:01.104498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:01.104937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:01.105642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:01.106091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:01.106514 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:01.243564 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'services' on Instance uuid a8351183-2123-48d0-9f68-c06ab2829d71 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:39:01.369258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e46470df-07b5-48dd-b644-7994294c07ed tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments/7fc3975c-c233-43c0-aa63-cdcf158a4760" status: 202 len: 0 microversion: 2.1 time: 0.276906 Oct 04 08:39:01.369899 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 257/458] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:39:01 2022] DELETE /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments/7fc3975c-c233-43c0-aa63-cdcf158a4760 => generated 0 bytes in 278 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:39:01.379110 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:01.381493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:01.389015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:01.389519 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:01.389990 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:01.390810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:01.391270 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:01.391695 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:01.432018 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9c6ead07-db4f-4950-8c30-871e312fcbea tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057552 Oct 04 08:39:01.432678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 258/459] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:39:01 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:02.071898 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:02.072827 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:02.081506 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.082220 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.082867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.084318 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.085086 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.085368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.229286 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:02.239053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ac9d5dcb-0816-41f8-ac9b-8e165f329d3a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.171357 Oct 04 08:39:02.239959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 259/460] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:02 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:02.444752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:02.447570 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:02.454705 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.455212 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.455647 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.456778 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.457251 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.458029 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.488730 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1d378c07-a98e-4667-b972-5fb5e4080982 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047898 Oct 04 08:39:02.489299 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 260/461] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:39:02 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 04 08:39:02.556017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:02.556520 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:02.563308 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.563794 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.564255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.564970 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.565424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.565850 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.599061 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:39:02.703025 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-644a8d6a-b92a-421f-8433-c7081740c780 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 204 len: 0 microversion: 2.1 time: 0.151690 Oct 04 08:39:02.703728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 261/462] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:39:02 2022] DELETE /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:39:02.713414 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:02.714169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:02.720908 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.721398 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.721841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.722581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:02.723029 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:02.723699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:02.961980 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:02.969923 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7f9bbe56-c4d0-4775-8dbf-ba9103e28038 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1680 microversion: 2.1 time: 0.260759 Oct 04 08:39:02.970505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 262/463] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:02 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1680 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:03.262499 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:03.263134 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:03.269845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:03.270331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:03.270792 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:03.271490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:03.271940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:03.272361 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:03.413962 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:03.424939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-555a52af-ff4b-4766-b87e-8d289d438a88 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.166116 Oct 04 08:39:03.425550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 263/464] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:03 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:03.993633 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:03.994304 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:04.002040 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:04.002533 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:04.002993 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:04.003742 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:04.004194 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:04.004614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:04.141976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:04.150498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7ed3dfa5-737b-4398-87a2-c543d833480f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1680 microversion: 2.1 time: 0.160790 Oct 04 08:39:04.151364 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 264/465] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:03 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1680 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:04.450354 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:04.450984 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:04.457820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:04.458299 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:04.458754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:04.459439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:04.459887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:04.460305 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:04.591948 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:04.601617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4c848553-18e1-4637-9fd1-71baba903c2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1724 microversion: 2.1 time: 0.155033 Oct 04 08:39:04.602202 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 265/466] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:04 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1724 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:04.726503 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-414578942", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "networks": [{"uuid": "6a117e0a-fcb5-404d-b62e-3e5b877ed8ef"}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:04.785223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] validate_networks() for [('6a117e0a-fcb5-404d-b62e-3e5b877ed8ef', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:39:05.423865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:39:05.424467 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:39:05.424870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:39:05.425358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:39:05.425829 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:39:05.436282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:05.436771 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:05.437481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:05.438196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:05.438674 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:05.439099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:05.458686 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Getting quotas for project f0b762e9e807416b925d8d34bb16fbf0. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:39:05.463649 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Getting quotas for user 9c42d5aec138420a90b87226dbe6dc20 and project f0b762e9e807416b925d8d34bb16fbf0. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:39:05.479273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:39:05.481595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:39:05.531211 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3161581b-9b49-4632-a991-bb7fa54ccf1f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.807851 Oct 04 08:39:05.537925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 266/467] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:39:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 810 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 04 08:39:05.537925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:05.537925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:05.545921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:05.546559 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:05.547004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:05.547899 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:05.548303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:05.548730 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:05.571472 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:05.632406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:05.644331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7cf756e2-8b89-44a9-b567-c6ffe4bb2433 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1613 microversion: 2.1 time: 0.110828 Oct 04 08:39:05.644937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 267/468] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:05 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1613 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:05.650797 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-061af56d-334a-4024-8ecf-7ce586656f8a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:05.651917 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-061af56d-334a-4024-8ecf-7ce586656f8a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:05.736815 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-061af56d-334a-4024-8ecf-7ce586656f8a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:05.751661 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-061af56d-334a-4024-8ecf-7ce586656f8a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1218 microversion: 2.1 time: 0.104332 Oct 04 08:39:05.754700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 268/469] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:05 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1218 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:05.760443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:05.761490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:05.769347 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:05.770247 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:05.770937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:05.771914 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:05.772594 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:05.773266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.044647 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:06.055084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3824ba8a-7039-4c29-85fd-c3874ea7740e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1701 microversion: 2.1 time: 0.298816 Oct 04 08:39:06.066748 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 269/470] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:05 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1701 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:06.081409 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-6711b21c-6f75-4be2-a929-605a7e00b309 req-575051f1-4479-4159-9e85-b4ec17202ef9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a8351183-2123-48d0-9f68-c06ab2829d71", "name": "network-vif-deleted", "tag": "4b6d91d8-8d57-437a-8e26-84b89843a14e"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:06.095732 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-6711b21c-6f75-4be2-a929-605a7e00b309 req-575051f1-4479-4159-9e85-b4ec17202ef9 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.097044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-6711b21c-6f75-4be2-a929-605a7e00b309 req-575051f1-4479-4159-9e85-b4ec17202ef9 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.098698 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-6711b21c-6f75-4be2-a929-605a7e00b309 req-575051f1-4479-4159-9e85-b4ec17202ef9 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.113488 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-6711b21c-6f75-4be2-a929-605a7e00b309 req-575051f1-4479-4159-9e85-b4ec17202ef9 service nova] Creating event network-vif-deleted:4b6d91d8-8d57-437a-8e26-84b89843a14e for instance a8351183-2123-48d0-9f68-c06ab2829d71 on hv-cinder-86020 Oct 04 08:39:06.119362 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-6711b21c-6f75-4be2-a929-605a7e00b309 req-575051f1-4479-4159-9e85-b4ec17202ef9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050982 Oct 04 08:39:06.120194 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 270/471] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:06.128681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d35dd8b5-7815-4651-b382-dc79771c5584 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:06.132103 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:06.139933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d35dd8b5-7815-4651-b382-dc79771c5584 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.141101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d35dd8b5-7815-4651-b382-dc79771c5584 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.141532 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d35dd8b5-7815-4651-b382-dc79771c5584 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.174646 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d35dd8b5-7815-4651-b382-dc79771c5584 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Going to try to start instance {{(pid=127142) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 04 08:39:06.228441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d35dd8b5-7815-4651-b382-dc79771c5584 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action" status: 202 len: 0 microversion: 2.1 time: 0.105722 Oct 04 08:39:06.228441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 271/472] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:39:06 2022] POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action => generated 0 bytes in 107 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:39:06.238633 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:06.240052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:06.251244 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.252694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.253219 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.254171 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.255272 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.255720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.447376 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:06.458128 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4848c1fc-a7d8-42ee-af2e-9709a835f2e7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1710 microversion: 2.1 time: 0.225135 Oct 04 08:39:06.459234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 272/473] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:06 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1710 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:06.669063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:06.669688 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:06.676754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.677333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.677899 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.678631 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.679114 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.679533 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.710043 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.710519 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.710977 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.711417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid a8351183-2123-48d0-9f68-c06ab2829d71 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:39:06.763363 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:06.792780 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6fa8414c-f7a0-479b-a1a5-f087b44d3ef0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 200 len: 1331 microversion: 2.1 time: 0.127456 Oct 04 08:39:06.794054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 273/474] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:06 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 1331 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:06.816300 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:06.817082 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:06.834471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.834997 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.835430 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.836136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:06.837254 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:06.837838 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:06.923914 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:06.932589 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c4997d99-b4d1-4267-a1d4-14ab020af066 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1214 microversion: 2.1 time: 0.136974 Oct 04 08:39:06.933754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 274/475] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:06 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1214 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:07.481720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:07.482359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:07.490200 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:07.490753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:07.491169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:07.491873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:07.492323 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:07.492839 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:07.629096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:07.637411 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3e06ebde-9971-4a67-a61c-5a7fd15e925b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1710 microversion: 2.1 time: 0.159660 Oct 04 08:39:07.637979 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 275/476] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:07 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1710 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:07.821519 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:07.822016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:07.828790 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:07.829461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:07.829903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:07.830672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:07.831326 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:07.831918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:07.847617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 HTTP/1.1 Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:39:07.849294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:39:07.850992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': 'a8351183-2123-48d0-9f68-c06ab2829d71'} Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:39:07.851960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:39:07.853319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:07.855604 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:39:07.856514 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-12d2f69e-10f9-4de7-87fd-0b37918146e6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 404 len: 111 microversion: 2.1 time: 0.039031 Oct 04 08:39:07.857101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 276/477] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:07 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:39:07.867019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cc749cd4-6c45-4daa-90f4-e907bcf867a8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '.version_select at 0x7f30a08d24c0>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:07.890520 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cc749cd4-6c45-4daa-90f4-e907bcf867a8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-607435801" status: 202 len: 0 microversion: 2.1 time: 0.027249 Oct 04 08:39:07.891816 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 277/478] 10.222.0.16 () {58 vars in 1218 bytes} [Tue Oct 4 08:39:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-607435801 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:39:07.958160 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:07.958860 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:07.966038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:07.966576 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:07.967021 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:07.967724 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:07.968283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:07.968985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:08.301846 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:08.310939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1069c92e-dfd8-418c-ae16-8563e44dfe29 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1276 microversion: 2.1 time: 0.356582 Oct 04 08:39:08.326979 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 278/479] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:07 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1276 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:08.595118 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d0ac9837-5b9c-47fb-9d9a-f9c7c26ea2d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: .version_select at 0x7f30a08d2f70>, body: {"keypair": {"name": "tempest-keypair-1383308230", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM9jXPTFOsGnt3e6SqXikzbMjVG3eByt9hOnTyrfDbY8SGkKs44k1J8CyjEI4iEBt4arJj8uls81WKYCZeEdJU/WDwCE2NAxxdaQzA/wQxAVw8UP1ieVEmdlIwkaMXmn3Q=="}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:08.614879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-d0ac9837-5b9c-47fb-9d9a-f9c7c26ea2d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:39:08.619147 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-d0ac9837-5b9c-47fb-9d9a-f9c7c26ea2d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:39:08.644937 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d0ac9837-5b9c-47fb-9d9a-f9c7c26ea2d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.053753 Oct 04 08:39:08.646704 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 279/480] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:39:08 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:08.664708 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:08.665885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:08.678330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:08.679364 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:08.679919 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:08.680933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:08.681504 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:08.682138 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:08.984267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:08.993066 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5b6e51f1-5cb4-4dad-8706-2c116555492c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1714 microversion: 2.1 time: 0.333890 Oct 04 08:39:08.993654 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 280/481] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:08 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1714 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:09.338610 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:09.339550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:09.346049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:09.346568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:09.347010 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:09.347718 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:09.348170 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:09.348597 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:09.599638 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:09.609790 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b47d5a7a-fb68-4ae6-96e7-c49b7f9c7d0e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.275589 Oct 04 08:39:09.614672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 281/482] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:09 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:09.825067 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-5030b732-0604-448a-9bdb-4af69029b036 req-a267ff4a-11da-49a2-b503-e9351065ad72 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "676277b4-479b-4158-ab82-141e10ce0337", "tag": "f6b61041-455e-49e4-b3da-963ee4f4ffe7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:09.845617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-5030b732-0604-448a-9bdb-4af69029b036 req-a267ff4a-11da-49a2-b503-e9351065ad72 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:09.846344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-5030b732-0604-448a-9bdb-4af69029b036 req-a267ff4a-11da-49a2-b503-e9351065ad72 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:09.847219 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-5030b732-0604-448a-9bdb-4af69029b036 req-a267ff4a-11da-49a2-b503-e9351065ad72 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:09.865391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-5030b732-0604-448a-9bdb-4af69029b036 req-a267ff4a-11da-49a2-b503-e9351065ad72 service nova] Creating event network-changed:f6b61041-455e-49e4-b3da-963ee4f4ffe7 for instance 676277b4-479b-4158-ab82-141e10ce0337 on hv-cinder-86020 Oct 04 08:39:09.870836 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-5030b732-0604-448a-9bdb-4af69029b036 req-a267ff4a-11da-49a2-b503-e9351065ad72 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052560 Oct 04 08:39:09.871513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 282/483] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:10.292571 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1107524263", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd380db5-2756-484c-8401-ea63df26e43d"}], "security_groups": [{"name": "tempest-securitygroup--466211208"}], "key_name": "tempest-keypair-1383308230", "user_data": "}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:10.416952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] validate_networks() for [('fd380db5-2756-484c-8401-ea63df26e43d', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:39:10.819426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:39:10.820048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:39:10.820328 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:39:10.820735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:39:10.821141 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:39:10.831158 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:10.831647 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:10.832087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:10.832817 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:10.833266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:10.833691 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:10.852497 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:39:10.856553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:39:10.868386 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:39:10.870838 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:39:10.934897 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4501629c-c39f-497b-b6f2-401d2771ccdc tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.647997 Oct 04 08:39:10.935562 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 283/484] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:39:10 2022] POST /compute/v2.1/servers => generated 401 bytes in 650 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 04 08:39:10.940651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:10.941274 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:10.948590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:10.949110 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:10.949515 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:10.950227 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:10.950810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:10.951150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:11.101359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:11.110227 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-20f019df-6f5d-4860-937e-0cccb18934e9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.173625 Oct 04 08:39:11.111140 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 284/485] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:10 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:11.115720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-194e1f12-a4a3-4a6a-8fb1-3c3dbe07c345 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:11.116324 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-194e1f12-a4a3-4a6a-8fb1-3c3dbe07c345 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:11.176543 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-194e1f12-a4a3-4a6a-8fb1-3c3dbe07c345 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:11.184520 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-194e1f12-a4a3-4a6a-8fb1-3c3dbe07c345 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1242 microversion: 2.1 time: 0.072332 Oct 04 08:39:11.185124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 285/486] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:11 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1242 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:12.146843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:12.147873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:12.154731 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:12.155215 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:12.155649 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:12.156348 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:12.157044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:12.157214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:12.613115 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:12.621852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7cfe2633-e49a-414d-9999-391b240a3a0f tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.479072 Oct 04 08:39:12.622449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 286/487] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:12 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:12.643461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:12.644105 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:12.651276 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:12.651725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:12.652298 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:12.653218 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:12.653705 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:12.654137 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:12.756954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:12.767500 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9b5cfb73-ffd0-4a50-a214-3dd84a5c78d0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1300 microversion: 2.1 time: 0.131627 Oct 04 08:39:12.768939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 287/488] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:12 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1300 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:13.650062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:13.650801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:13.657978 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:13.658468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:13.658932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:13.659643 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:13.660098 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:13.660529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:13.902139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:13.916535 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-66dbb674-1d08-474d-8dd7-94b037c51a34 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.270284 Oct 04 08:39:13.921737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 288/489] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:13 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:13.927129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:13.927842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:13.934964 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:13.935452 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:13.935890 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:13.937275 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:13.937629 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:13.938060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:13.957335 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:14.178076 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:14.186313 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7a32308b-2e5f-4051-a976-842c677b04f9 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.263095 Oct 04 08:39:14.198007 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 289/490] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:13 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:14.754473 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-9e889986-c59e-4452-a881-029cebb26859 req-0617371c-709b-4573-90f0-9f31bc28b46a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "255e6091-63f4-4381-89bc-d6b720ecb68e", "tag": "24f06c20-7f60-43e3-94eb-e8a55284c173"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:14.773002 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e889986-c59e-4452-a881-029cebb26859 req-0617371c-709b-4573-90f0-9f31bc28b46a service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:14.773468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e889986-c59e-4452-a881-029cebb26859 req-0617371c-709b-4573-90f0-9f31bc28b46a service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:14.773902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e889986-c59e-4452-a881-029cebb26859 req-0617371c-709b-4573-90f0-9f31bc28b46a service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:14.787794 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-9e889986-c59e-4452-a881-029cebb26859 req-0617371c-709b-4573-90f0-9f31bc28b46a service nova] Creating event network-changed:24f06c20-7f60-43e3-94eb-e8a55284c173 for instance 255e6091-63f4-4381-89bc-d6b720ecb68e on hv-cinder-86020 Oct 04 08:39:14.793564 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-9e889986-c59e-4452-a881-029cebb26859 req-0617371c-709b-4573-90f0-9f31bc28b46a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046670 Oct 04 08:39:14.794417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 290/491] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:14.942121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:14.942761 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:14.949091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:14.949572 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:14.950003 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:14.950752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:14.951202 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:14.951652 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:15.165309 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:15.174051 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b5732aaa-9740-41bd-97b8-84df949e5709 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.236276 Oct 04 08:39:15.174898 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 291/492] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:14 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:15.214242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:15.215230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:15.243168 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:15.243674 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:15.244242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:15.244947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:15.245366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:15.245788 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:15.495905 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:15.504568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-53d717af-044b-403f-811b-a066f1c1bac8 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.295233 Oct 04 08:39:15.509592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 292/493] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:15 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:16.217665 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-1cbeb72a-f681-40ed-b39e-f2f7e7f923a7 req-d52d37cc-1904-4b93-8c27-33fe825601e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "676277b4-479b-4158-ab82-141e10ce0337", "name": "network-vif-plugged", "status": "completed", "tag": "f6b61041-455e-49e4-b3da-963ee4f4ffe7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:16.229347 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-1cbeb72a-f681-40ed-b39e-f2f7e7f923a7 req-d52d37cc-1904-4b93-8c27-33fe825601e6 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:16.229841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-1cbeb72a-f681-40ed-b39e-f2f7e7f923a7 req-d52d37cc-1904-4b93-8c27-33fe825601e6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:16.230252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-1cbeb72a-f681-40ed-b39e-f2f7e7f923a7 req-d52d37cc-1904-4b93-8c27-33fe825601e6 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:16.242817 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-1cbeb72a-f681-40ed-b39e-f2f7e7f923a7 req-d52d37cc-1904-4b93-8c27-33fe825601e6 service nova] Creating event network-vif-plugged:f6b61041-455e-49e4-b3da-963ee4f4ffe7 for instance 676277b4-479b-4158-ab82-141e10ce0337 on hv-cinder-86020 Oct 04 08:39:16.247823 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-1cbeb72a-f681-40ed-b39e-f2f7e7f923a7 req-d52d37cc-1904-4b93-8c27-33fe825601e6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034012 Oct 04 08:39:16.248623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 293/494] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:39:16.264231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:16.265383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:16.272645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:16.273129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:16.273561 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:16.274266 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:16.274752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:16.275169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:16.411689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:16.419614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-20fd267b-bc6b-4959-8bf8-2060246a825a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.159673 Oct 04 08:39:16.420193 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 294/495] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:16 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:16.528279 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:16.528839 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:16.535569 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:16.536056 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:16.536486 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:16.537177 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:16.537640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:16.538044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:16.671515 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:16.679418 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4b6e170a-b8cc-495a-b92c-b765222950c0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.155061 Oct 04 08:39:16.680000 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 295/496] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:16 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:17.444079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:17.444779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:17.451497 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:17.452842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:17.453264 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:17.454124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:17.454600 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:17.455030 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:17.642705 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:17.650495 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f95f34e7-b6b0-4fda-b373-71d25172786b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.210226 Oct 04 08:39:17.651092 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 296/497] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:17 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:17.702754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:17.703517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:17.709497 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:17.709974 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:17.710402 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:17.711123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:17.711576 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:17.711997 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:17.851742 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:17.860527 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cb7d7ceb-9df5-4227-b031-741717fb9703 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.161770 Oct 04 08:39:17.861122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 297/498] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:17 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:18.672903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:18.673595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:18.680152 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:18.680633 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:18.681060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:18.681755 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:18.682195 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:18.682642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:18.827174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:18.836235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-dcc826b1-e832-43da-8156-961eca4162a3 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1318 microversion: 2.1 time: 0.167310 Oct 04 08:39:18.843102 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 298/499] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:18 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1318 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:18.884777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:18.885396 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:18.891642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:18.892121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:18.892552 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:18.893234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:18.893675 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:18.894088 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:19.040615 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:19.048781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b661717e-fb59-400f-9fe8-57020d3c7346 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.169472 Oct 04 08:39:19.049383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 299/500] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:18 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:19.861037 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:19.861701 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:19.868367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:19.868848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:19.869277 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:19.869976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:19.870422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:19.870887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:20.032911 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:20.041675 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-01eab8c5-aa1f-4285-a4c1-66f7c70a9ad5 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1508 microversion: 2.1 time: 0.185552 Oct 04 08:39:20.042272 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 300/501] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:19 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1508 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:20.071718 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:20.072523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:20.079830 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:20.080980 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:20.081448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:20.082440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:20.082933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:20.083365 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:20.287135 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:20.300512 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7050e061-9617-46fd-894d-c21dd5c69264 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.232006 Oct 04 08:39:20.302612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 301/502] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:20 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:20.314324 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-683f8cb1-afc7-4e2b-8f18-478ec1325808 req-0818d7c1-b9fb-424f-b0f8-a959cc7ae7bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "255e6091-63f4-4381-89bc-d6b720ecb68e", "name": "network-vif-plugged", "status": "completed", "tag": "24f06c20-7f60-43e3-94eb-e8a55284c173"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:20.328745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-683f8cb1-afc7-4e2b-8f18-478ec1325808 req-0818d7c1-b9fb-424f-b0f8-a959cc7ae7bc service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:20.329719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-683f8cb1-afc7-4e2b-8f18-478ec1325808 req-0818d7c1-b9fb-424f-b0f8-a959cc7ae7bc service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:20.330594 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-683f8cb1-afc7-4e2b-8f18-478ec1325808 req-0818d7c1-b9fb-424f-b0f8-a959cc7ae7bc service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:20.348080 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-683f8cb1-afc7-4e2b-8f18-478ec1325808 req-0818d7c1-b9fb-424f-b0f8-a959cc7ae7bc service nova] Creating event network-vif-plugged:24f06c20-7f60-43e3-94eb-e8a55284c173 for instance 255e6091-63f4-4381-89bc-d6b720ecb68e on hv-cinder-86020 Oct 04 08:39:20.355623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-683f8cb1-afc7-4e2b-8f18-478ec1325808 req-0818d7c1-b9fb-424f-b0f8-a959cc7ae7bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046123 Oct 04 08:39:20.356907 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 302/503] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:21.091185 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.096253 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-35cd2bf3-6af9-4c88-a6a8-6d0afc1a568e tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Calling method '.version_select at 0x7f30a08d2f70>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:21.116507 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-35cd2bf3-6af9-4c88-a6a8-6d0afc1a568e tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-391951627" status: 202 len: 0 microversion: 2.1 time: 0.026208 Oct 04 08:39:21.117599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 303/504] 10.222.0.16 () {58 vars in 1251 bytes} [Tue Oct 4 08:39:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-391951627 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:39:21.322248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=20 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.326427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:21.327265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:21.329153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:21.335723 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:21.336210 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:21.336652 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:21.337368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:21.337822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:21.338249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:21.476762 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:21.485401 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-530a0669-39d5-4321-a2f9-83f8dafc58f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.163859 Oct 04 08:39:21.486130 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 304/505] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:21 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:22.509711 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:22.510454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:22.519091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:22.519591 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:22.520067 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:22.520852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:22.521315 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:22.521745 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:22.708346 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:22.729506 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5784f6fa-f92e-4182-b72e-6075d161bca1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1367 microversion: 2.1 time: 0.223656 Oct 04 08:39:22.730102 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 305/506] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:22 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1367 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:23.754101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:23.754806 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:23.761575 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:23.762098 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:23.762498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:23.763244 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:23.763998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:23.764941 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:23.949885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:23.958459 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4a12f795-cef5-4d85-8a4c-d64f128da661 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1557 microversion: 2.1 time: 0.209039 Oct 04 08:39:23.959248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 306/507] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:23 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1557 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:23.984226 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:23.984760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:23.990960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:23.991441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:23.992034 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:23.992739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:23.993189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:23.993608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:24.138944 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0e932df0-012b-491b-9227-65d4376f67e5 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.159219 Oct 04 08:39:24.140216 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 307/508] 10.222.0.16 () {58 vars in 1275 bytes} [Tue Oct 4 08:39:23 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-interface => generated 284 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:25.987019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-709181a2-e123-47b1-8131-7373c7cc40e2 req-720f5702-f6c6-4173-8626-79ca9e59b372 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "255e6091-63f4-4381-89bc-d6b720ecb68e", "tag": "24f06c20-7f60-43e3-94eb-e8a55284c173"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:25.996932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-709181a2-e123-47b1-8131-7373c7cc40e2 req-720f5702-f6c6-4173-8626-79ca9e59b372 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:25.997416 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-709181a2-e123-47b1-8131-7373c7cc40e2 req-720f5702-f6c6-4173-8626-79ca9e59b372 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:25.997845 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-709181a2-e123-47b1-8131-7373c7cc40e2 req-720f5702-f6c6-4173-8626-79ca9e59b372 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:26.012375 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-709181a2-e123-47b1-8131-7373c7cc40e2 req-720f5702-f6c6-4173-8626-79ca9e59b372 service nova] Creating event network-changed:24f06c20-7f60-43e3-94eb-e8a55284c173 for instance 255e6091-63f4-4381-89bc-d6b720ecb68e on hv-cinder-86020 Oct 04 08:39:26.019179 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-709181a2-e123-47b1-8131-7373c7cc40e2 req-720f5702-f6c6-4173-8626-79ca9e59b372 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036752 Oct 04 08:39:26.020283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 308/509] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:39:27.199017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "30635ed0-8bd4-45c4-b688-9288e30ee564"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:27.202931 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:27.210204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:27.210969 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:27.211611 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:27.223419 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:27.224214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:27.224837 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:27.391613 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:27.576032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a6971fce-b73e-423f-b0cd-7d0da24a3498 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.381081 Oct 04 08:39:27.576032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 309/510] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:39:27 2022] POST /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments => generated 194 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:39:32.676046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:32.676713 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:32.683036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:32.683515 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:32.683947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:32.684674 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:32.685119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:32.685539 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:32.716784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bf21be5b-0110-4df6-9062-cbf826e98607 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments/bc71647d-9cff-4147-9916-cad71db53183" status: 200 len: 194 microversion: 2.20 time: 0.046820 Oct 04 08:39:32.717565 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 310/511] 10.222.0.16 () {60 vars in 1454 bytes} [Tue Oct 4 08:39:32 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments/bc71647d-9cff-4147-9916-cad71db53183 => generated 194 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:39:32.789885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:32.790378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:32.797229 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:32.797904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:32.798518 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:32.799553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:32.800164 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:32.800784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:32.890659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lazy-loading 'services' on Instance uuid 54819895-e219-4341-8f1e-5543b7357315 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:39:32.965089 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-48453ca1-6076-4fa8-b7fe-3d794e10036f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments/bc71647d-9cff-4147-9916-cad71db53183" status: 202 len: 0 microversion: 2.20 time: 0.178913 Oct 04 08:39:32.965842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 311/512] 10.222.0.16 () {60 vars in 1457 bytes} [Tue Oct 4 08:39:32 2022] DELETE /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments/bc71647d-9cff-4147-9916-cad71db53183 => generated 0 bytes in 181 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:39:32.975791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:32.978071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:32.985442 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:32.985920 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:32.986369 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:32.987109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:32.987772 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:32.988150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:33.033292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d9e8a05b-f1fe-4db6-91f9-34eb98e32a12 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061575 Oct 04 08:39:33.033876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 312/513] 10.222.0.16 () {60 vars in 1343 bytes} [Tue Oct 4 08:39:32 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:39:34.046758 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:34.048967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:34.055986 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:34.056478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:34.056924 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:34.057638 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:34.058088 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:34.058507 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:34.091301 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2b6050c7-fcd2-449b-99bb-0484da77aa9a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049667 Oct 04 08:39:34.091874 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 313/514] 10.222.0.16 () {60 vars in 1343 bytes} [Tue Oct 4 08:39:34 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:39:35.106328 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:35.110618 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:35.119027 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:35.119522 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:35.119973 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:35.120678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:35.121133 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:35.121710 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:35.169834 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b612502b-de3b-482b-be80-5cee3272e9e0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.069196 Oct 04 08:39:35.171964 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 314/515] 10.222.0.16 () {60 vars in 1343 bytes} [Tue Oct 4 08:39:35 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:39:35.272725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:35.275084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:35.282538 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:35.283058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:35.283490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:35.284214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:35.284683 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:35.285108 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:35.315435 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'flavor' on Instance uuid f5ecca0c-50c6-4218-a34e-f4ec70d2686e {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:39:35.464949 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9fffd283-f7aa-49d8-bed6-d7f5262234fb tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/action" status: 200 len: 32 microversion: 2.1 time: 0.196580 Oct 04 08:39:35.465577 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 315/516] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:39:35 2022] POST /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/action => generated 32 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 04 08:39:35.476483 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:35.477188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:35.484360 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:35.484865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:35.485300 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:35.486010 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:35.486464 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:35.486972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:35.713253 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:35.723488 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7f73d5d8-7bcf-4925-b7e6-51047aac23d8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.251151 Oct 04 08:39:35.724073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 316/517] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:35 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:36.184004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:36.187440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:36.197147 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.198007 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.198848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:36.200214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.201074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.202457 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:36.239136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6a111123-4fe2-410e-b93b-1b504848827f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.058909 Oct 04 08:39:36.239740 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 317/518] 10.222.0.16 () {60 vars in 1343 bytes} [Tue Oct 4 08:39:36 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315/os-volume_attachments => generated 25 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:36.307358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:36.307884 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:36.314022 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.314514 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.314992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:36.315700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.316153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.316583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:36.345022 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:39:36.442617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b2ea573e-d50e-44bd-9115-863f5c9098da tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 204 len: 0 microversion: 2.20 time: 0.138437 Oct 04 08:39:36.443612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 318/519] 10.222.0.16 () {60 vars in 1280 bytes} [Tue Oct 4 08:39:36 2022] DELETE /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 04 08:39:36.454718 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:36.455426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:36.461881 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.462475 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.463104 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:36.463826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.464341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.464765 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:36.622420 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:36.632021 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:36.633474 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:36.634667 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5f617b14-3564-4ed2-9280-7dbcb64b76a4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.185526 Oct 04 08:39:36.635238 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 319/520] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:36 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:39:36.640520 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0a058186-d154-49a0-a7c9-483b67c57aa1", "device": "/dev/sdb"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:36.643386 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:36.649828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.650346 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.650821 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:36.651519 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:36.651965 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:36.652382 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:38.113507 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:UPDATE=5,nova_cell1:SELECT=91,nova_cell1:INSERT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.255425 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e7830c3b-6491-4cdf-ab7e-1fbdd44ad86a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "POST /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.618959 Oct 04 08:39:38.256049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 320/521] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:39:36 2022] POST /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337/os-volume_attachments => generated 194 bytes in 1620 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:38.261330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:38.262143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:38.269149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:38.269640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:38.270073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:38.271035 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:38.271491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:38.271915 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:38.431293 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:38.440197 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a6af48f5-033e-4738-a9ed-e342f62fb860 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.182898 Oct 04 08:39:38.440948 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 321/522] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:38 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:38.445569 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:38.446344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:38.453394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:38.453950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:38.454378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:38.455336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:38.455865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:38.456276 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:38.678055 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:38.686699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:38.688283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:38.689560 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5fa794b2-8bb4-4540-b1af-c1f414196d74 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1724 microversion: 2.20 time: 0.247857 Oct 04 08:39:38.690145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 322/523] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:38 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1724 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:39:39.466623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:39.467855 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:39.482061 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:39.483716 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:39.484230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:39.485264 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:39.486188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:39.486188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:39.669419 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:39.683551 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c6b1cc30-1761-49f5-8deb-45946d0c80e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.220907 Oct 04 08:39:39.687487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 323/524] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:39 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:39:39.692639 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-b0d45350-007e-41b6-a29c-5af2e0633a83 req-f83b71d4-c6da-4d7b-975d-caddcd85f900 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54819895-e219-4341-8f1e-5543b7357315", "name": "network-vif-deleted", "tag": "159277a9-e8de-47c4-813b-8de31279c18f"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:39.710306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-b0d45350-007e-41b6-a29c-5af2e0633a83 req-f83b71d4-c6da-4d7b-975d-caddcd85f900 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:39.710817 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-b0d45350-007e-41b6-a29c-5af2e0633a83 req-f83b71d4-c6da-4d7b-975d-caddcd85f900 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:39.711252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-b0d45350-007e-41b6-a29c-5af2e0633a83 req-f83b71d4-c6da-4d7b-975d-caddcd85f900 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:39.730766 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-b0d45350-007e-41b6-a29c-5af2e0633a83 req-f83b71d4-c6da-4d7b-975d-caddcd85f900 service nova] Creating event network-vif-deleted:159277a9-e8de-47c4-813b-8de31279c18f for instance 54819895-e219-4341-8f1e-5543b7357315 on hv-cinder-86020 Oct 04 08:39:39.738050 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-b0d45350-007e-41b6-a29c-5af2e0633a83 req-f83b71d4-c6da-4d7b-975d-caddcd85f900 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048900 Oct 04 08:39:39.738689 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 324/525] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:39.772443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:39.773156 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:39.780870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:39.781947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:39.782454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:39.783259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:39.783706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:39.784342 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:39.817650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:39.818130 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:39.818584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:39.819020 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lazy-loading 'fault' on Instance uuid 54819895-e219-4341-8f1e-5543b7357315 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:39:39.862395 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:39.871101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:39.872594 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:39.875301 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bec07892-92be-4c7a-8095-32431e032eff tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 200 len: 1370 microversion: 2.20 time: 0.135283 Oct 04 08:39:39.882902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 325/526] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:39 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 1370 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:39:40.709032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:40.709710 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:40.716950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:40.717586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:40.718017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:40.718784 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:40.719232 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:40.719649 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:40.857530 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:40.866053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-59aaaa9d-4d3d-4ba7-8454-d1e9d2ea6e57 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.160866 Oct 04 08:39:40.866657 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 326/527] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:40 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:40.921887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:40.922541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:40.928944 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:40.929435 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:40.929868 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:40.930612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:40.931065 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:40.931483 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:40.946729 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] HTTP exception thrown: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Request method failure captured: Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 HTTP/1.1 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BE8CUPPt8UTo8wxyjiitahg4WwA74AGkiBCEnsdRaJLKQNG2l38LIokg_eUW1qaBKdMxDHY2oHljwCLhRSfHbTrbeIwxfV0dPgvGFHeTebAxT7VFsODm3KBGGbp9WSsMikMmG2HpFYxWvFVsRLrymAAwp5gI4QbbJxZDYhCBs0Ph0 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Openstack-Nova-Api-Version: 2.20 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:39:40.948429 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e938efbd6804a86946d80ae92e88b16", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 87ca9fc2e013477cad3a3de8e8811274 Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:39:40.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': '54819895-e219-4341-8f1e-5543b7357315'} Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:39:40.951155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:39:40.953417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:39:40.954992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Returning 404 to user: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:39:40.954992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a491cbdb-76f1-4f33-ad8f-d6ba74719a03 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 404 len: 111 microversion: 2.20 time: 0.033128 Oct 04 08:39:40.954992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 327/528] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:40 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 04 08:39:40.960608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-15f2e6bf-6a47-44d2-ba70-5a572bdabc43 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '.version_select at 0x7f30a0a5b790>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:40.984070 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-15f2e6bf-6a47-44d2-ba70-5a572bdabc43 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-499785230" status: 204 len: 0 microversion: 2.20 time: 0.026648 Oct 04 08:39:40.984685 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 328/529] 10.222.0.16 () {60 vars in 1259 bytes} [Tue Oct 4 08:39:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-499785230 => generated 0 bytes in 29 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 04 08:39:41.674258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3cb45edd-0568-41bf-88e5-5cb413ec68a7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Action: 'create', calling method: .version_select at 0x7f30a08d2040>, body: {"keypair": {"name": "tempest-keypair-596417858", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG4D6x77OEbVoKVYMOicQEyZn0jPuSbn8K4Nku4hpM0IIfdnIBGA25AlOiudGQeGyanoppKpKISMGi+dAOMbFyE58qgYpKe/XssTvY8sbd3fIrcBDT4tyYEo8luhIeYB3A=="}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:41.691951 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-3cb45edd-0568-41bf-88e5-5cb413ec68a7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for project 3e938efbd6804a86946d80ae92e88b16. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:39:41.695947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-3cb45edd-0568-41bf-88e5-5cb413ec68a7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for user 87ca9fc2e013477cad3a3de8e8811274 and project 3e938efbd6804a86946d80ae92e88b16. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:39:41.717665 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3cb45edd-0568-41bf-88e5-5cb413ec68a7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.047773 Oct 04 08:39:41.718735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 329/530] 10.222.0.16 () {62 vars in 1200 bytes} [Tue Oct 4 08:39:41 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 49 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Oct 04 08:39:41.890792 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:41.891490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:41.897906 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:41.898502 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:41.898918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:41.899636 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:41.900265 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:41.900687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:42.108294 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:42.117206 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-76ea5b4c-eeb9-4afa-9439-8cba7c6a7ecc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.230641 Oct 04 08:39:42.117802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 330/531] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:41 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:43.144856 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:43.146667 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:43.155977 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:43.157008 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:43.157492 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:43.158866 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:43.159345 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:43.159903 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:43.318361 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:43.329495 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8a4ab378-dfb9-49ed-8897-7509c252479e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.191564 Oct 04 08:39:43.330047 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 331/532] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:43 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:43.335416 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-92155583", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "730a9b11-4347-4415-adec-96bd4e59e83e"}], "security_groups": [{"name": "tempest-securitygroup--1995877511"}], "key_name": "tempest-keypair-596417858", "user_data": "}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:43.456052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] validate_networks() for [('730a9b11-4347-4415-adec-96bd4e59e83e', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:39:43.849771 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:39:43.850184 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:39:43.850688 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:39:43.851118 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:39:43.851556 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:39:43.861184 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:43.861733 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:43.862082 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:43.863156 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:43.863293 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:43.863699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:43.881113 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for project 3e938efbd6804a86946d80ae92e88b16. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:39:43.885810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Getting quotas for user 87ca9fc2e013477cad3a3de8e8811274 and project 3e938efbd6804a86946d80ae92e88b16. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:39:43.897840 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:39:43.900221 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:39:43.949405 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c2568a6b-7cd4-4cb7-92b2-d596c7153cf0 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.618068 Oct 04 08:39:43.950066 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 332/533] 10.222.0.16 () {62 vars in 1188 bytes} [Tue Oct 4 08:39:43 2022] POST /compute/v2.1/servers => generated 402 bytes in 619 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 04 08:39:43.967063 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-37b4412f-51a7-4438-865e-747c082510f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:43.968550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-37b4412f-51a7-4438-865e-747c082510f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:44.032781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-37b4412f-51a7-4438-865e-747c082510f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:44.041487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-37b4412f-51a7-4438-865e-747c082510f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:44.043024 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-37b4412f-51a7-4438-865e-747c082510f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:44.044177 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-37b4412f-51a7-4438-865e-747c082510f7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1279 microversion: 2.20 time: 0.084535 Oct 04 08:39:44.044964 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 333/534] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:43 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1279 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:39:44.353744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:44.354369 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:44.363083 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:44.363936 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:44.364409 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:44.365159 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:44.365617 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:44.366038 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:44.531439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:44.540911 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a0f87563-618f-4261-ac14-7455930d3eed tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.191274 Oct 04 08:39:44.542167 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 334/535] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:44 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:45.071932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:45.072543 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:45.075674 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:45.079707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:45.080734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:45.081120 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:45.082166 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:45.082772 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:45.083395 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:45.164340 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:45.177663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:45.179448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:45.181106 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b7ebedc9-f8c2-49f8-a150-c7e9d6667a32 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1331 microversion: 2.20 time: 0.113383 Oct 04 08:39:45.181676 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 335/536] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:45 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1331 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:45.565760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:45.566405 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:45.574009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:45.574486 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:45.574965 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:45.575636 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:45.576086 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:45.576621 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:45.737545 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:45.747587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5a7bd863-4c42-493b-b6c7-963a47bed330 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.185868 Oct 04 08:39:45.748440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 336/537] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:45 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:46.211006 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:46.211623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:46.219398 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:46.219925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:46.220352 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:46.221145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:46.221623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:46.222111 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:46.565619 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:46.574370 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:46.575898 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:46.577772 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-628ff0c3-4008-47b0-babe-ce319ff78805 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1337 microversion: 2.20 time: 0.370922 Oct 04 08:39:46.578587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 337/538] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:46 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1337 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:46.771940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:46.772680 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:46.779697 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:46.780213 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:46.780658 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:46.781389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:46.781924 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:46.782357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:47.005907 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:47.021172 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5f169a0b-d0ed-487c-a5cf-abbf2c39e9d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.253083 Oct 04 08:39:47.022317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 338/539] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:46 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:47.603119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:47.603782 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:47.611018 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:47.611502 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:47.611943 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:47.612645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:47.613096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:47.613519 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:47.767859 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:47.776153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:47.777618 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:47.778776 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-57f1f41f-b7e3-468d-8c8b-04dc9df0ed23 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.179778 Oct 04 08:39:47.779368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 339/540] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:47 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:48.047343 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:48.048100 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:48.054895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:48.055438 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:48.055919 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:48.056684 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:48.057187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:48.057651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:48.223720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:48.243032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5f599b7f-0666-4d03-a1cd-affb6eab9001 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.199806 Oct 04 08:39:48.244708 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 340/541] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:48 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:48.806873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:48.807952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:48.817627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:48.818207 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:48.818855 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:48.819700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:48.820204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:48.820866 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:48.824569 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=97,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:49.160738 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:49.175240 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:49.179711 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:49.183127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5fe99cc9-d730-4d83-a810-79749dcd2ff4 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.380527 Oct 04 08:39:49.198744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 341/542] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:48 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:39:49.198744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-fdcf3907-ad03-4259-81f9-d0f3b874cb46 req-0a7726af-91e6-43e0-8586-ba82e2947a27 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "72e91abf-2688-4fce-a3ea-42775bee6210", "tag": "8c389990-706b-42b4-8cba-e3f6d810fbb6"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:49.214802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-fdcf3907-ad03-4259-81f9-d0f3b874cb46 req-0a7726af-91e6-43e0-8586-ba82e2947a27 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:49.216650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-fdcf3907-ad03-4259-81f9-d0f3b874cb46 req-0a7726af-91e6-43e0-8586-ba82e2947a27 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:49.223579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-fdcf3907-ad03-4259-81f9-d0f3b874cb46 req-0a7726af-91e6-43e0-8586-ba82e2947a27 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:49.254494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-fdcf3907-ad03-4259-81f9-d0f3b874cb46 req-0a7726af-91e6-43e0-8586-ba82e2947a27 service nova] Creating event network-changed:8c389990-706b-42b4-8cba-e3f6d810fbb6 for instance 72e91abf-2688-4fce-a3ea-42775bee6210 on hv-cinder-86020 Oct 04 08:39:49.262328 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-fdcf3907-ad03-4259-81f9-d0f3b874cb46 req-0a7726af-91e6-43e0-8586-ba82e2947a27 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074893 Oct 04 08:39:49.263350 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 342/543] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:39:49.273798 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:49.274625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:49.282314 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:49.283017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:49.283464 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:49.284177 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:49.284624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:49.285043 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:49.677854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:49.687076 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-40f0f71f-575a-4dd0-9c30-d0ff6726c6c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.419571 Oct 04 08:39:49.687880 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 343/544] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:49 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:50.234172 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:50.235190 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:50.241793 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:50.242307 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:50.242777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:50.243505 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:50.243959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:50.244389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:50.419221 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:50.434391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:50.438122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:50.440566 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9ab75756-87ae-41f8-8c23-e875f871b05e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.210161 Oct 04 08:39:50.441488 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 344/545] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:50 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:50.715233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:50.716367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:50.727449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:50.728281 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:50.729031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:50.730230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:50.731057 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:50.731814 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:50.881558 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:50.892648 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-39f708fb-ac33-4f2b-9cfb-18c41deb7dea tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.184374 Oct 04 08:39:50.893322 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 345/546] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:50 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:51.468509 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:51.469163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:51.475774 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:51.476257 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:51.476694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:51.477399 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:51.477888 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:51.478311 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:51.623939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:51.632834 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:51.634298 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:51.635469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7c8223b9-402f-453a-b0a6-93386553fcb7 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.171619 Oct 04 08:39:51.636044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 346/547] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:51 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:51.917460 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:51.918122 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:51.924715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:51.925252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:51.925782 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:51.926896 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:51.927453 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:51.927915 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:52.088269 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:52.103267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c40e973d-b954-4aa8-aa44-4f1aadaa88e5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.189333 Oct 04 08:39:52.104239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 347/548] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:51 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:52.661972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:52.662642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:52.669362 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:52.670720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:52.671131 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:52.672020 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:52.672473 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:52.672895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:52.814650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:52.823856 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:52.825424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:52.827196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ce6f647b-77eb-4fae-8cd9-2596434ea433 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.169213 Oct 04 08:39:52.827765 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 348/549] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:52 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:53.129865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:53.130493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:53.136886 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:53.137378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:53.137815 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:53.139475 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:53.139673 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:53.140015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:53.292260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:53.303048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d05a6f1f-a3f3-43fe-bc20-9fcf832752e0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.176907 Oct 04 08:39:53.304005 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 349/550] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:53 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:53.854579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:53.855732 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:53.864337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:53.864971 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:53.865764 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:53.866681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:53.867145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:53.867568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:53.871339 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:54.038316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=12 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:54.128286 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:54.142642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:54.145019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:54.146825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7fa7a8bf-aefe-47e2-8dd1-463e953beecd tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.296134 Oct 04 08:39:54.155513 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 350/551] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:53 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:39:54.345622 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:54.346887 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:54.355226 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:54.356712 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:54.357075 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:54.359182 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:54.360420 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:54.361308 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:54.581149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:54.594327 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a762b7df-542e-44b8-9f17-ee39c6ce8d05 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.254540 Oct 04 08:39:54.595036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 351/552] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:54 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:55.173935 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:55.174630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:55.181469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:55.182007 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:55.182439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:55.183171 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:55.183614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:55.184032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:55.360877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:55.369426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:55.370991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:55.372224 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-66f0d81c-d674-4dba-a461-f8af910276af tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.202586 Oct 04 08:39:55.373069 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 352/553] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:55 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:55.618515 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:55.619104 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:55.625503 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:55.625981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:55.626408 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:55.627123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:55.627598 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:55.628013 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:55.757507 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:55.766511 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c7401f9a-0335-44c2-8346-2374e468ef5e tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.151883 Oct 04 08:39:55.767114 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 353/554] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:55 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:56.397980 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:56.398732 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:56.406237 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:56.406807 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:56.407189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:56.407883 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:56.408329 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:56.408752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:56.554093 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:56.563597 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:56.565058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:56.566239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-251d0421-b997-4684-b476-aa3d76177c82 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.172228 Oct 04 08:39:56.566824 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 354/555] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:56 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:56.790754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:56.791290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:56.797393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:56.797877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:56.798316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:56.799046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:56.799498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:56.799918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:56.946155 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:56.956132 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a9d0a5b5-d0b2-4596-91ac-05b04c8d5287 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.169275 Oct 04 08:39:56.956719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 355/556] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:56 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:57.592763 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:57.593404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:57.600307 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:57.600788 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:57.601217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:57.601988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:57.602381 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:57.602906 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:57.745650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:57.755870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:57.757340 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:57.758510 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a3b3bbe3-5e0e-4fb1-878f-732cd365e79e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.169978 Oct 04 08:39:57.759120 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 356/557] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:57 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:57.908118 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-0c87b2ad-8a87-4d28-97b9-0a90511985f1 req-77485f39-efc1-4ba0-a0a8-5826c6497ff3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "72e91abf-2688-4fce-a3ea-42775bee6210", "name": "network-vif-plugged", "status": "completed", "tag": "8c389990-706b-42b4-8cba-e3f6d810fbb6"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:39:57.918322 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0c87b2ad-8a87-4d28-97b9-0a90511985f1 req-77485f39-efc1-4ba0-a0a8-5826c6497ff3 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:57.918915 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0c87b2ad-8a87-4d28-97b9-0a90511985f1 req-77485f39-efc1-4ba0-a0a8-5826c6497ff3 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:57.919379 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0c87b2ad-8a87-4d28-97b9-0a90511985f1 req-77485f39-efc1-4ba0-a0a8-5826c6497ff3 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:57.937520 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-0c87b2ad-8a87-4d28-97b9-0a90511985f1 req-77485f39-efc1-4ba0-a0a8-5826c6497ff3 service nova] Creating event network-vif-plugged:8c389990-706b-42b4-8cba-e3f6d810fbb6 for instance 72e91abf-2688-4fce-a3ea-42775bee6210 on hv-cinder-86020 Oct 04 08:39:57.944700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-0c87b2ad-8a87-4d28-97b9-0a90511985f1 req-77485f39-efc1-4ba0-a0a8-5826c6497ff3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039853 Oct 04 08:39:57.945974 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 357/558] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:39:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:39:57.980187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:57.981029 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:57.988879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:57.989490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:57.990019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:57.990935 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:57.991492 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:57.992078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:58.178606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:58.188412 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-caaee9a2-a75d-471b-9568-4dcc2c7ac25f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.212572 Oct 04 08:39:58.189004 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 358/559] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:57 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:58.193932 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:58.194437 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:58.200515 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:58.201044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:58.201483 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:58.202189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:58.202671 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:58.203094 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:58.205959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=100 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:58.298685 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lazy-loading 'services' on Instance uuid 9606a52d-3c5d-4287-afb7-c84f5e245c9a {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:39:58.378131 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-045e4752-2623-487e-9dec-bbdce80edfbc tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments/4418b356-105a-4f4d-a378-da2da99c3d91" status: 202 len: 0 microversion: 2.1 time: 0.187892 Oct 04 08:39:58.378767 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 359/560] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:39:58 2022] DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments/4418b356-105a-4f4d-a378-da2da99c3d91 => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:39:58.785764 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:58.786448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:58.794124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:58.795591 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:58.796447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:58.797428 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:58.798017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:58.798449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:58.946596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:58.955522 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:58.956971 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:58.958088 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b5dafdef-57e1-4c2f-97c3-9471d8cd1277 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.177163 Oct 04 08:39:58.958682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 360/561] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:58 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:39:59.212412 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:59.212970 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:59.219672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:59.220174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:59.220587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:59.221284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:59.221732 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:59.222157 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:59.366093 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:39:59.376203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e7078119-6de3-45ae-82d4-659561a69e35 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.167606 Oct 04 08:39:59.377072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 361/562] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:39:59 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:39:59.983047 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:39:59.984372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:39:59.991510 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:59.992035 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:59.992471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:39:59.993199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:39:59.993623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:39:59.994053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:00.138061 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:00.150422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:00.152496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:00.153668 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1d6b3acc-46b8-4b0a-ac7b-2d032e911c88 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1405 microversion: 2.20 time: 0.174598 Oct 04 08:40:00.154842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 362/563] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:39:59 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1405 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:40:00.406198 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:00.407158 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:00.418316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:00.419189 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:00.420979 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:00.422239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:00.422945 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:00.423675 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:00.587697 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:00.597323 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-728ee446-0e3a-4bca-8b5a-a13565fb42e3 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.198566 Oct 04 08:40:00.597915 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 363/564] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:00 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:00.602584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-918c8a32-3ff6-4b35-ad7c-16de73c75001 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:00.610049 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-918c8a32-3ff6-4b35-ad7c-16de73c75001 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:00.610538 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-918c8a32-3ff6-4b35-ad7c-16de73c75001 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:00.610994 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-918c8a32-3ff6-4b35-ad7c-16de73c75001 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:00.637434 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-918c8a32-3ff6-4b35-ad7c-16de73c75001 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Going to try to stop instance {{(pid=127142) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 04 08:40:00.694443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-918c8a32-3ff6-4b35-ad7c-16de73c75001 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action" status: 202 len: 0 microversion: 2.1 time: 0.095124 Oct 04 08:40:00.695168 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 364/565] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:40:00 2022] POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:40:00.708985 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:00.710223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:00.719823 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:00.720400 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:00.721080 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:00.722303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:00.723055 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:00.723597 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:00.883016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:00.897812 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9c5bc009-c96d-4e88-969f-095f25171df5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.198021 Oct 04 08:40:00.898417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 365/566] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:00 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:01.184253 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:01.184886 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:01.192660 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.193148 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.193584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:01.194285 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.194769 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.195195 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:01.343978 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:01.360000 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:01.361485 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:01.362696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bf742b82-dcbc-4c3a-9937-3ef7c85f7b0d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1597 microversion: 2.20 time: 0.182360 Oct 04 08:40:01.363279 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 366/567] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:40:01 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1597 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 04 08:40:01.392373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:01.392996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:01.400308 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.400801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.401238 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:01.401958 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.402407 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.402865 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:01.491541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-22f27b79-d686-4ed7-8646-322fe5fedf15 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.105523 Oct 04 08:40:01.492251 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 367/568] 10.222.0.16 () {60 vars in 1316 bytes} [Tue Oct 4 08:40:01 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/os-interface => generated 284 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 04 08:40:01.623663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:01.624287 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:01.633921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.634716 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.635364 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:01.636631 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.637443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.638188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:01.934652 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:01.944961 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cb53f474-1445-4626-8452-a86a19180b6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.325002 Oct 04 08:40:01.945998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 368/569] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:01 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:01.954536 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:01.955663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:01.962403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.963220 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.963728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:01.964593 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:01.965104 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:01.966306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:02.194680 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:02.204599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a1403dff-d0a9-439d-9481-bba76ed28b31 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.257106 Oct 04 08:40:02.207770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 369/570] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:01 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:02.427342 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:02.428078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:02.435453 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:02.436153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:02.436608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:02.437340 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:02.437891 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:02.438324 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:02.533750 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lazy-loading 'services' on Instance uuid 676277b4-479b-4158-ab82-141e10ce0337 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:40:02.616818 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-87d8c9e1-8be1-42e7-b32e-ff64b53e792a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337/os-volume_attachments/0a058186-d154-49a0-a7c9-483b67c57aa1" status: 202 len: 0 microversion: 2.1 time: 0.193403 Oct 04 08:40:02.617957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 370/571] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:40:02 2022] DELETE /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337/os-volume_attachments/0a058186-d154-49a0-a7c9-483b67c57aa1 => generated 0 bytes in 196 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:40:02.976069 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:02.976776 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:02.983614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:02.984106 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:02.984545 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:02.985256 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:02.985708 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:02.986136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:03.145066 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:03.159196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-002ccb11-b0e1-46ae-a73a-26e903158df5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.188224 Oct 04 08:40:03.178964 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 371/572] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:02 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:03.178964 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-8c0d091d-e9a9-426c-99a1-d67fbd5529ed req-45a947de-3d28-4a55-9637-e29c08a10e96 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "72e91abf-2688-4fce-a3ea-42775bee6210", "tag": "8c389990-706b-42b4-8cba-e3f6d810fbb6"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:03.178964 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-8c0d091d-e9a9-426c-99a1-d67fbd5529ed req-45a947de-3d28-4a55-9637-e29c08a10e96 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:03.178964 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-8c0d091d-e9a9-426c-99a1-d67fbd5529ed req-45a947de-3d28-4a55-9637-e29c08a10e96 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:03.179950 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-8c0d091d-e9a9-426c-99a1-d67fbd5529ed req-45a947de-3d28-4a55-9637-e29c08a10e96 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:03.193474 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-8c0d091d-e9a9-426c-99a1-d67fbd5529ed req-45a947de-3d28-4a55-9637-e29c08a10e96 service nova] Creating event network-changed:8c389990-706b-42b4-8cba-e3f6d810fbb6 for instance 72e91abf-2688-4fce-a3ea-42775bee6210 on hv-cinder-86020 Oct 04 08:40:03.199686 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-8c0d091d-e9a9-426c-99a1-d67fbd5529ed req-45a947de-3d28-4a55-9637-e29c08a10e96 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038678 Oct 04 08:40:03.200346 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 372/573] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:40:03.235834 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:03.236550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:03.243107 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:03.243632 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:03.244039 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:03.244788 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:03.245241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:03.247435 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:03.519469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:03.539586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f307e2bb-6f03-4f0e-aa87-c0e519c10742 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.309899 Oct 04 08:40:03.540877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 373/574] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:03 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:03.770001 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:03.770508 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:03.777184 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:03.777683 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:03.778123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:03.779798 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:03.780009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:03.780652 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:03.803649 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:03.823632 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:40:03.944896 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a9a6f648-5a7b-429e-b26d-5027f285fe2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 204 len: 0 microversion: 2.1 time: 0.178599 Oct 04 08:40:03.945540 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 374/575] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:40:03 2022] DELETE /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:40:03.955446 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:03.956072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:03.962215 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:03.962739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:03.963176 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:03.963886 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:03.964338 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:03.964754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:04.133752 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:04.144587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b721ec0c-d664-44c3-b73f-9809acfc9f86 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1560 microversion: 2.1 time: 0.194028 Oct 04 08:40:04.146001 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 375/576] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:03 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:04.193758 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:04.194961 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:04.205437 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:04.206288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:04.206697 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:04.207568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:04.208768 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:04.209213 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:04.383442 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:04.397050 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-941ef831-1b89-4947-bb95-c5a29645d429 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.210472 Oct 04 08:40:04.398159 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 376/577] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:04 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:04.571953 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:04.573333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:04.584099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:04.585233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:04.586042 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:04.587100 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:04.587651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:04.588153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:04.771192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:04.781726 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-377894af-b319-4102-adc7-50ba06f55cd1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.217012 Oct 04 08:40:04.782361 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 377/578] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:04 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:05.185470 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:05.186368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:05.192808 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:05.193286 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:05.193709 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:05.194405 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:05.194874 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:05.195286 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:05.348081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:05.356422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-62f3e4d1-e994-47f1-ba04-caebcdfbb60a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1514 microversion: 2.1 time: 0.175475 Oct 04 08:40:05.357087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 378/579] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:05 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1514 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:05.423235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:05.423751 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:05.429677 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:05.430149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:05.430596 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:05.431288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:05.431734 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:05.432153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:05.572162 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:05.581096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7851a349-ef64-49d4-8bb8-8d451134e6f5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.162459 Oct 04 08:40:05.581686 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 379/580] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:05 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:05.805526 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:05.806057 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:05.812949 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:05.813428 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:05.813852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:05.814541 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:05.815053 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:05.815469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:05.953237 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:05.961663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-92fb36aa-e19c-452f-80b6-edcc431d0116 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.160094 Oct 04 08:40:05.962250 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 380/581] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:05 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:06.379875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:06.380757 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:06.387308 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:06.387706 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:06.388165 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:06.389412 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:06.389959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:06.390463 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:06.746373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:06.756026 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-81eb48fc-2bbe-4f6a-957b-c946b73b5e72 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1514 microversion: 2.1 time: 0.379999 Oct 04 08:40:06.756716 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 381/582] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:06 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1514 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:06.761405 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:06.762115 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:06.764573 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:06.770650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:06.771144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:06.771831 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:06.772670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:06.773439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:06.774052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:07.045796 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:07.054718 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3511bd0e-38ab-426e-8037-29ad0a5d52d2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.296842 Oct 04 08:40:07.055341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 382/583] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:06 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:07.059992 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:07.060659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:07.067125 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:07.067618 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:07.068055 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:07.068766 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:07.069220 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:07.069690 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:07.292273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:07.303679 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5040cf8e-4750-4b75-956d-34540aab61b5 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.247282 Oct 04 08:40:07.304585 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 383/584] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:07 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:07.400847 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-c9abb795-a0e4-41c1-a760-21e7ef322c69 req-fe03f720-406a-4515-931c-4d057f1bd7c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "676277b4-479b-4158-ab82-141e10ce0337", "name": "network-vif-deleted", "tag": "f6b61041-455e-49e4-b3da-963ee4f4ffe7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:07.415754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-c9abb795-a0e4-41c1-a760-21e7ef322c69 req-fe03f720-406a-4515-931c-4d057f1bd7c2 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:07.416216 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-c9abb795-a0e4-41c1-a760-21e7ef322c69 req-fe03f720-406a-4515-931c-4d057f1bd7c2 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:07.416655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-c9abb795-a0e4-41c1-a760-21e7ef322c69 req-fe03f720-406a-4515-931c-4d057f1bd7c2 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:07.429072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-c9abb795-a0e4-41c1-a760-21e7ef322c69 req-fe03f720-406a-4515-931c-4d057f1bd7c2 service nova] Creating event network-vif-deleted:f6b61041-455e-49e4-b3da-963ee4f4ffe7 for instance 676277b4-479b-4158-ab82-141e10ce0337 on hv-cinder-86020 Oct 04 08:40:07.434281 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-c9abb795-a0e4-41c1-a760-21e7ef322c69 req-fe03f720-406a-4515-931c-4d057f1bd7c2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037552 Oct 04 08:40:07.434942 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 384/585] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:07.782696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:07.783433 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:07.790390 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:07.790904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:07.791339 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:07.792045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:07.792496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:07.792919 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:07.827078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:07.827564 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:07.828003 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:07.828451 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lazy-loading 'fault' on Instance uuid 676277b4-479b-4158-ab82-141e10ce0337 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:40:07.871341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:07.880313 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-578d91ad-c6a5-4f1d-9d9c-466bf3c2d59c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1308 microversion: 2.1 time: 0.102373 Oct 04 08:40:07.880895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 385/586] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:07 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1308 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:08.081019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:08.081724 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:08.088104 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:08.088590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:08.089025 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:08.089728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:08.090187 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:08.090635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:08.307753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:08.323408 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a12db65d-409d-4897-b628-56df4d1accff tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1735 microversion: 2.1 time: 0.246178 Oct 04 08:40:08.324427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 386/587] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:08 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1735 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:08.337816 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:08.338465 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:08.345942 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:08.346503 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:08.346981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:08.347690 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:08.348148 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:08.348560 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:08.507388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:08.521730 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8fd5f70c-c7a0-49cb-b953-24b8a1e0a900 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.187443 Oct 04 08:40:08.522732 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 387/588] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:08 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:08.905087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:08.905806 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:08.912655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:08.913130 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:08.913573 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:08.914306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:08.914795 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:08.915222 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:08.955979 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:08.956469 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:08.957424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:08.957424 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lazy-loading 'fault' on Instance uuid 676277b4-479b-4158-ab82-141e10ce0337 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:40:09.007164 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:09.015325 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3944b254-f07e-4593-9b99-029b09434649 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 200 len: 1308 microversion: 2.1 time: 0.114702 Oct 04 08:40:09.015928 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 388/589] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:08 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 1308 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:09.348510 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:09.349129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:09.355372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:09.355857 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:09.356296 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:09.357553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:09.357999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:09.358435 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:09.375336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:09.677127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:09.685939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-86c853e0-6380-416b-ae05-ccd3c84d3fc0 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1715 microversion: 2.1 time: 0.341433 Oct 04 08:40:09.686527 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 389/590] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:09 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1715 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:09.691234 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:09.691851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:09.700365 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:09.700976 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:09.701542 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:09.702477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:09.703174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:09.703731 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:09.880325 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:09.889808 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6b2062c8-506b-44cd-a604-d77c730e4a8d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.202052 Oct 04 08:40:09.890597 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 390/591] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:09 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:09.895358 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:09.895852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:09.902175 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:09.902692 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:09.903136 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:09.904375 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:09.904829 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:09.905310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:09.998000 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Cannot 'detach_volume' instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e while it is in vm_state rescued Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:40:09.999895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:40:10.001482 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Cannot 'detach_volume' instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e while it is in vm_state rescued Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:40:10.002303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e while it is in vm_state rescued Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e while it is in vm_state rescued Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.003606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 409 to user: Cannot 'detach_volume' instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e while it is in vm_state rescued {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:40:10.004747 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9f7253f5-bf92-4e43-b8e0-3bf771cea49b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 409 len: 152 microversion: 2.1 time: 0.109756 Oct 04 08:40:10.004747 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 391/592] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:40:09 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 152 bytes in 111 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 04 08:40:10.010387 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:10.010940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:10.017939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.018428 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.018880 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:10.019574 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.020025 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.020443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:10.092911 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-df2241b9-6cc8-4a1f-8302-0b76e6a1497b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "POST /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/action" status: 202 len: 0 microversion: 2.1 time: 0.085907 Oct 04 08:40:10.093605 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 392/593] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:40:10 2022] POST /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/action => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:40:10.099200 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:10.099891 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 676277b4-479b-4158-ab82-141e10ce0337] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:10.106274 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.106796 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.107291 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:10.108028 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.108431 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.108848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:10.123957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 676277b4-479b-4158-ab82-141e10ce0337 could not be found. Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 HTTP/1.1 Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:40:10.125870 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 676277b4-479b-4158-ab82-141e10ce0337 could not be found. Oct 04 08:40:10.127612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': '676277b4-479b-4158-ab82-141e10ce0337'} Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 676277b4-479b-4158-ab82-141e10ce0337 could not be found. Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:40:10.128449 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 676277b4-479b-4158-ab82-141e10ce0337 could not be found. Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 676277b4-479b-4158-ab82-141e10ce0337 could not be found. Oct 04 08:40:10.129635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:10.131615 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 676277b4-479b-4158-ab82-141e10ce0337 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:40:10.131615 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a9c0de20-ce98-4447-a00e-c41425865929 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337" status: 404 len: 111 microversion: 2.1 time: 0.032467 Oct 04 08:40:10.131615 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 393/594] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:10 2022] GET /compute/v2.1/servers/676277b4-479b-4158-ab82-141e10ce0337 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:40:10.134496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:10.135157 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:10.145497 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.146102 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.146823 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:10.147650 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.148160 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.148687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:10.380828 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:10.391874 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4a826fdc-be75-44a5-8824-018e382b3f2f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1723 microversion: 2.1 time: 0.263065 Oct 04 08:40:10.392707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 394/595] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:10 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1723 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:10.914592 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:10.915259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:10.921968 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.922458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.922921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:10.923620 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:10.924075 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:10.924497 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:11.063871 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:11.071977 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-5e3be9f7-2253-4b93-aab8-b7512f905e2a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.161536 Oct 04 08:40:11.072558 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 395/596] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:10 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:11.416399 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:11.417018 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:11.423909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:11.424412 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:11.424851 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:11.425572 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:11.426026 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:11.426454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:11.570999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:11.585055 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6bc2822d-1e80-4134-99c4-7c1a5ee0ebe2 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1723 microversion: 2.1 time: 0.172291 Oct 04 08:40:11.585993 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 396/597] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:11 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:12.098125 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:12.098940 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:12.106010 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:12.106506 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:12.107670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:12.108728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:12.109196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:12.109634 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:12.257642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:12.267517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c984f4ed-5065-47a2-91ae-62a977318930 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.174072 Oct 04 08:40:12.268332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 397/598] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:12 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:12.616141 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:12.616758 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:12.624010 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:12.624746 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:12.625196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:12.625904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:12.626365 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:12.626819 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:12.762068 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:12.771638 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4e687ee4-6939-4e93-988b-b14dec7e9e38 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1723 microversion: 2.1 time: 0.159482 Oct 04 08:40:12.772217 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 398/599] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:12 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:12.927881 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:12.928373 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:12.934450 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:12.935573 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:12.936178 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:12.936883 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:12.937333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:12.937753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:12.966009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8ea68ee2-4581-498e-8778-41abffe779f4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/ips" status: 200 len: 150 microversion: 2.1 time: 0.041831 Oct 04 08:40:12.966805 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 399/600] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:40:12 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:13.295169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:13.296773 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:13.317150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:13.319241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:13.328562 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:13.330196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:13.331191 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:13.339277 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:13.790790 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:13.799094 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-31b862bf-69d2-49e0-aa87-61b67927019b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1678 microversion: 2.1 time: 0.509256 Oct 04 08:40:13.799668 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 400/601] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:13 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1678 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:13.804447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:13.805058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:13.812820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:13.813628 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:13.814161 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:13.815260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:13.815805 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:13.816317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:13.973983 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:13.983268 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3271428f-db9c-45d5-8ba7-476366d8567a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1723 microversion: 2.1 time: 0.182560 Oct 04 08:40:13.984108 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 401/602] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:13 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1723 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:14.334117 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2a1c677a-e68b-4049-9874-03334a29625a"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:14.336605 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:14.343561 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:14.344041 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:14.344690 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:14.345394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:14.345848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:14.346269 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.261027 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1eab4390-71fe-41ae-aa94-3a0adebd5d69 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.930798 Oct 04 08:40:15.261664 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 402/603] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:40:14 2022] POST /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments => generated 194 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:15.267443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:15.268137 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:15.274883 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:15.275208 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:15.275635 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.276336 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:15.276780 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:15.277199 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.444990 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:15.453989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-76b87fe6-8f67-4377-ac65-133a5a8ea832 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1655 microversion: 2.1 time: 0.191126 Oct 04 08:40:15.454610 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 403/604] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:15 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1655 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:15.459313 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:15.459933 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:15.466141 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:15.467753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:15.467753 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.468350 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:15.468801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:15.469227 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.623692 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:15.627479 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:15.636627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4baa5fc2-fac2-4ff9-9f99-69e6daa7b196 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1723 microversion: 2.1 time: 0.180949 Oct 04 08:40:15.637225 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 404/605] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:15 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1723 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:15.643422 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1159811f-0f59-4c3b-b5a4-376192e3b47e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:15.650285 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1159811f-0f59-4c3b-b5a4-376192e3b47e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:15.650757 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1159811f-0f59-4c3b-b5a4-376192e3b47e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:15.651191 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1159811f-0f59-4c3b-b5a4-376192e3b47e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.680526 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1159811f-0f59-4c3b-b5a4-376192e3b47e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Going to try to start instance {{(pid=127142) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 04 08:40:15.726439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1159811f-0f59-4c3b-b5a4-376192e3b47e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action" status: 202 len: 0 microversion: 2.1 time: 0.087968 Oct 04 08:40:15.727090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 405/606] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:40:15 2022] POST /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:40:15.735938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:15.736633 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:15.742900 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:15.743384 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:15.743820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.744516 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:15.744966 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:15.745387 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:15.884928 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:15.895310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d8d9398c-f190-4cec-ac7c-c3a1d58b9305 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1664 microversion: 2.1 time: 0.163772 Oct 04 08:40:15.895967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 406/607] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:15 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1664 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:16.664124 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:16.665061 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:16.672078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:16.672569 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:16.673039 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:16.673750 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:16.674205 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:16.674663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:16.813197 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:16.821793 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-30a675f7-d02a-4279-b4a3-7d236f5cbefa tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1723 microversion: 2.1 time: 0.161520 Oct 04 08:40:16.822368 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 407/608] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:16 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:16.920087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:16.920712 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:16.926797 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:16.927284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:16.927714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:16.928406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:16.928853 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:16.929272 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:17.083223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:17.091645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-08408ccf-fa49-4bd3-bc62-33b36fda6dc0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1664 microversion: 2.1 time: 0.175625 Oct 04 08:40:17.092231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 408/609] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:16 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1664 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:17.845672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:17.847334 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:17.853943 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:17.854817 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:17.854973 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:17.855613 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:17.856071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:17.856499 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:18.011493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:18.031651 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9ea295cf-8ccb-4d4e-91ca-1fbdf804b14f tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1729 microversion: 2.1 time: 0.189271 Oct 04 08:40:18.033854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 409/610] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:17 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1729 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:18.056271 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:18.056761 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:18.063796 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:18.064282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:18.064768 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:18.065389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:18.065836 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:18.066255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:18.157320 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-62a9b187-5f3a-4ebe-8513-43e782c420b9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.104959 Oct 04 08:40:18.158077 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 410/611] 10.222.0.16 () {58 vars in 1275 bytes} [Tue Oct 4 08:40:18 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-interface => generated 284 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:18.163019 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:18.163622 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:18.169767 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:18.170400 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:18.171024 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:18.171769 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:18.172259 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:18.172733 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:18.566761 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:18.576975 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7851acf4-320e-44d9-bab3-2a4a521eff9e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1664 microversion: 2.1 time: 0.417830 Oct 04 08:40:18.577571 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 411/612] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:18 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1664 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:18.582681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-200803823", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "networks": [{"uuid": "6a117e0a-fcb5-404d-b62e-3e5b877ed8ef"}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:18.644116 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] validate_networks() for [('6a117e0a-fcb5-404d-b62e-3e5b877ed8ef', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:40:18.727919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:40:18.728528 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:40:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:40:20.284506 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:40:20.285044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:40:20.285934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:40:20.286359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:40:20.286840 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:40:20.297392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:20.297877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:20.298317 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:20.299109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:20.300274 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:20.300741 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:20.320699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Getting quotas for project f0b762e9e807416b925d8d34bb16fbf0. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:40:20.326892 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Getting quotas for user 9c42d5aec138420a90b87226dbe6dc20 and project f0b762e9e807416b925d8d34bb16fbf0. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:40:20.339078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:40:20.342127 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:40:20.395640 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-dca32d4b-3385-4553-a266-c55ac4883434 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.816835 Oct 04 08:40:20.403121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 412/613] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:40:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 1819 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:40:20.405330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:20.406702 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:20.415779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:20.416844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:20.417789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:20.419028 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:20.420458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:20.421250 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:20.642434 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:20.651476 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a77037e6-458e-446b-bb48-4d46e47102c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1664 microversion: 2.1 time: 0.253262 Oct 04 08:40:20.652676 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 413/614] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:20 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1664 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:20.662095 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-dce04c29-2d46-411b-9781-9d45f7d0d1a7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:20.665780 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-dce04c29-2d46-411b-9781-9d45f7d0d1a7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:20.804440 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-dce04c29-2d46-411b-9781-9d45f7d0d1a7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:20.814378 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-dce04c29-2d46-411b-9781-9d45f7d0d1a7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1218 microversion: 2.1 time: 0.160897 Oct 04 08:40:20.818443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 414/615] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:20 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1218 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:20.823534 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-0cd61125-765c-4624-b40a-13403e2b2888 req-66401385-531f-4e8a-aad2-db6d43981411 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5ecca0c-50c6-4218-a34e-f4ec70d2686e", "tag": "84c7f948-82c8-4e44-86f1-d8ace0e90f47"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:20.844367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0cd61125-765c-4624-b40a-13403e2b2888 req-66401385-531f-4e8a-aad2-db6d43981411 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:20.845188 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0cd61125-765c-4624-b40a-13403e2b2888 req-66401385-531f-4e8a-aad2-db6d43981411 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:20.846098 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-0cd61125-765c-4624-b40a-13403e2b2888 req-66401385-531f-4e8a-aad2-db6d43981411 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:20.857957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-0cd61125-765c-4624-b40a-13403e2b2888 req-66401385-531f-4e8a-aad2-db6d43981411 service nova] Creating event network-changed:84c7f948-82c8-4e44-86f1-d8ace0e90f47 for instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e on hv-cinder-86020 Oct 04 08:40:20.863310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-0cd61125-765c-4624-b40a-13403e2b2888 req-66401385-531f-4e8a-aad2-db6d43981411 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043298 Oct 04 08:40:20.866777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 415/616] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:40:21.684324 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:21.686342 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:21.693416 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:21.694033 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:21.694825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:21.695407 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:21.695821 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:21.696153 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:21.960185 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:21.978030 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-f4457646-b70c-4205-bcd2-e509340aa4b7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1668 microversion: 2.1 time: 0.297948 Oct 04 08:40:21.979101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 416/617] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:21 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1668 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:21.984939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:21.985714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:21.992837 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:21.993720 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:21.994270 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:21.995255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:21.995960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:21.996942 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:22.076904 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:22.085759 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-db8a59fd-ca81-4bb1-b676-b80c3ddc3fd2 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1270 microversion: 2.1 time: 0.105155 Oct 04 08:40:22.086300 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 417/618] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:21 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1270 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:22.901287 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-d6a73ee8-6ac2-4143-b46a-193fae3fbb90 req-6c65ba56-f256-4c90-b9e0-6a756c663ac5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5ecca0c-50c6-4218-a34e-f4ec70d2686e", "tag": "84c7f948-82c8-4e44-86f1-d8ace0e90f47"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:22.924854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-d6a73ee8-6ac2-4143-b46a-193fae3fbb90 req-6c65ba56-f256-4c90-b9e0-6a756c663ac5 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:22.925873 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-d6a73ee8-6ac2-4143-b46a-193fae3fbb90 req-6c65ba56-f256-4c90-b9e0-6a756c663ac5 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:22.926468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-d6a73ee8-6ac2-4143-b46a-193fae3fbb90 req-6c65ba56-f256-4c90-b9e0-6a756c663ac5 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:22.943518 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-d6a73ee8-6ac2-4143-b46a-193fae3fbb90 req-6c65ba56-f256-4c90-b9e0-6a756c663ac5 service nova] Creating event network-changed:84c7f948-82c8-4e44-86f1-d8ace0e90f47 for instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e on hv-cinder-86020 Oct 04 08:40:22.950451 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-d6a73ee8-6ac2-4143-b46a-193fae3fbb90 req-6c65ba56-f256-4c90-b9e0-6a756c663ac5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052969 Oct 04 08:40:22.951695 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 418/619] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:23.110485 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:23.111430 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:23.118479 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:23.119040 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:23.119468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:23.120179 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:23.120625 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:23.121436 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:23.218150 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:23.228568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-269eb64a-0716-4e09-890a-e0a7f546ca5a tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1276 microversion: 2.1 time: 0.123165 Oct 04 08:40:23.229164 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 419/620] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:23 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1276 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:24.261382 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:24.262050 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:24.269662 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:24.270143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:24.270587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:24.271288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:24.271727 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:24.272139 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:24.485455 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:24.493554 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-393557c3-df0c-45a2-accb-dacfaedf4b2e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1318 microversion: 2.1 time: 0.236069 Oct 04 08:40:24.494143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 420/621] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:24 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1318 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:24.980410 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-ee7eb995-96f4-4294-b75b-094560c9031a req-258df135-1ca3-4131-91f2-edb672d68a3b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a212616-19a8-409e-8dc3-0fe11b28ac76", "tag": "f7276a7a-f322-4f2a-94f2-80a317bcf537"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:24.991334 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-ee7eb995-96f4-4294-b75b-094560c9031a req-258df135-1ca3-4131-91f2-edb672d68a3b service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:24.991931 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-ee7eb995-96f4-4294-b75b-094560c9031a req-258df135-1ca3-4131-91f2-edb672d68a3b service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:24.994400 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-ee7eb995-96f4-4294-b75b-094560c9031a req-258df135-1ca3-4131-91f2-edb672d68a3b service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:25.007917 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-ee7eb995-96f4-4294-b75b-094560c9031a req-258df135-1ca3-4131-91f2-edb672d68a3b service nova] Creating event network-changed:f7276a7a-f322-4f2a-94f2-80a317bcf537 for instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 on hv-cinder-86020 Oct 04 08:40:25.013968 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-ee7eb995-96f4-4294-b75b-094560c9031a req-258df135-1ca3-4131-91f2-edb672d68a3b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037370 Oct 04 08:40:25.015458 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 421/622] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:40:25.522822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:25.523561 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:25.525507 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:25.532668 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:25.533546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:25.534329 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:25.535448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:25.536058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:25.536485 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:25.683031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:25.692475 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1037affd-bcf3-4d2d-8cf6-6bf3792a5bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1318 microversion: 2.1 time: 0.179480 Oct 04 08:40:25.693066 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 422/623] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:25 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1318 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:26.316883 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2a1c677a-e68b-4049-9874-03334a29625a"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:26.319278 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:26.325582 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.325975 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.326404 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.327129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.327573 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.327996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.411935 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Invalid volume: volume 2a1c677a-e68b-4049-9874-03334a29625a is already attached to instances: 255e6091-63f4-4381-89bc-d6b720ecb68e Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: POST /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments HTTP/1.1 Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 74 Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:40:26.413626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {"volumeAttachment": {"volumeId": "2a1c677a-e68b-4049-9874-03334a29625a"}} Oct 04 08:40:26.415367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Invalid volume: volume 2a1c677a-e68b-4049-9874-03334a29625a is already attached to instances: 255e6091-63f4-4381-89bc-d6b720ecb68e Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'server_id': '255e6091-63f4-4381-89bc-d6b720ecb68e', 'body': {'volumeAttachment': {'volumeId': '2a1c677a-e68b-4049-9874-03334a29625a'}}} Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 2a1c677a-e68b-4049-9874-03334a29625a is already attached to instances: 255e6091-63f4-4381-89bc-d6b720ecb68e Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:26.416252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 2a1c677a-e68b-4049-9874-03334a29625a is already attached to instances: 255e6091-63f4-4381-89bc-d6b720ecb68e Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:26.417413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 2a1c677a-e68b-4049-9874-03334a29625a is already attached to instances: 255e6091-63f4-4381-89bc-d6b720ecb68e Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 400 to user: Invalid volume: volume 2a1c677a-e68b-4049-9874-03334a29625a is already attached to instances: 255e6091-63f4-4381-89bc-d6b720ecb68e {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0e6c715d-aeff-4f6d-a908-8f918cda6335 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.102257 Oct 04 08:40:26.418630 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 423/624] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:40:26 2022] POST /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments => generated 174 bytes in 103 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 04 08:40:26.486998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:26.487487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:26.493553 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.494037 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.494473 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.495202 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.495659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.496078 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.512448 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:26.587310 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'services' on Instance uuid 255e6091-63f4-4381-89bc-d6b720ecb68e {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:40:26.671327 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1b66103d-eb83-46f3-aaf5-73d805eca7a1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments/2a1c677a-e68b-4049-9874-03334a29625a" status: 202 len: 0 microversion: 2.1 time: 0.187913 Oct 04 08:40:26.672131 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 424/625] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:40:26 2022] DELETE /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments/2a1c677a-e68b-4049-9874-03334a29625a => generated 0 bytes in 190 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:40:26.682758 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:26.685537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:26.693686 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.694185 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.694660 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.695584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.696196 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.696663 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.735116 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1942a21c-6fa8-4ba4-902e-a88cee4a35bd tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057714 Oct 04 08:40:26.735715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 425/626] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:40:26 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:26.740807 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:26.741417 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:26.747906 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.748466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.748814 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.749519 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:26.750223 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:26.750670 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:26.972227 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:26.980627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-63f048b4-073e-4a94-b932-2bdf05ff3bd9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1318 microversion: 2.1 time: 0.243869 Oct 04 08:40:26.981195 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 426/627] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:26 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1318 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:27.748991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:27.751612 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:27.758303 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:27.758825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:27.759258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:27.759948 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:27.760394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:27.760805 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:27.796472 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-ca78014e-9c2e-4e9c-b5e3-a37c501ac71d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052006 Oct 04 08:40:27.797048 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 427/628] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:40:27 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 04 08:40:27.870954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:27.871451 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:27.879062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:27.880466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:27.880466 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:27.880653 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:27.881073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:27.881492 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:27.910854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:40:28.007249 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-43ffa980-2c01-4e40-a260-0c6090496d68 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 204 len: 0 microversion: 2.1 time: 0.139851 Oct 04 08:40:28.008119 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 428/629] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:40:27 2022] DELETE /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:40:28.014282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:28.015311 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:28.021860 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:28.022361 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:28.022839 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:28.023552 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:28.024094 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:28.024525 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:28.202398 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:28.210326 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-23b25ede-5817-4f6f-be49-9c44cc279a7b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1318 microversion: 2.1 time: 0.200981 Oct 04 08:40:28.210957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 429/630] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:28 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1318 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:28.215566 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:28.217331 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:28.228366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:28.228866 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:28.229298 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:28.230045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:28.230498 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:28.230957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:28.373728 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:28.382154 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-49469841-424a-49a8-8948-701bf3eb11e7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1682 microversion: 2.1 time: 0.170130 Oct 04 08:40:28.382781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 430/631] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:28 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1682 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:29.236455 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:29.237105 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:29.243642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:29.244120 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:29.244548 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:29.245242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:29.245687 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:29.246099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:29.388825 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:29.396644 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-6f813025-6378-441b-91b2-ec72f43e0b50 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1318 microversion: 2.1 time: 0.165093 Oct 04 08:40:29.397215 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 431/632] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:29 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1318 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:29.405305 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:29.406729 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:29.416332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:29.417238 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:29.418045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:29.419350 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:29.420129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:29.420925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:29.571988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:29.580491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-88b1056e-9fa5-4137-a6b0-905f0162fa56 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1682 microversion: 2.1 time: 0.179130 Oct 04 08:40:29.581087 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 432/633] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:29 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1682 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:29.739208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:40:29.739777 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:40:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:40:30.098955 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-389e8456-a18a-4c51-97c0-24860c02c998 req-7488ec03-0432-4051-ab24-892de5b4a96d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a212616-19a8-409e-8dc3-0fe11b28ac76", "name": "network-vif-plugged", "status": "completed", "tag": "f7276a7a-f322-4f2a-94f2-80a317bcf537"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:30.113916 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-389e8456-a18a-4c51-97c0-24860c02c998 req-7488ec03-0432-4051-ab24-892de5b4a96d service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:30.114425 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-389e8456-a18a-4c51-97c0-24860c02c998 req-7488ec03-0432-4051-ab24-892de5b4a96d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:30.114947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-389e8456-a18a-4c51-97c0-24860c02c998 req-7488ec03-0432-4051-ab24-892de5b4a96d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:30.136555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-389e8456-a18a-4c51-97c0-24860c02c998 req-7488ec03-0432-4051-ab24-892de5b4a96d service nova] Creating event network-vif-plugged:f7276a7a-f322-4f2a-94f2-80a317bcf537 for instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 on hv-cinder-86020 Oct 04 08:40:30.147493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-389e8456-a18a-4c51-97c0-24860c02c998 req-7488ec03-0432-4051-ab24-892de5b4a96d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051572 Oct 04 08:40:30.148494 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 433/634] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:40:30.424031 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:30.425198 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:30.427487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:30.434237 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:30.434791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:30.435230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:30.435934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:30.436375 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:30.436967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:30.721822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:30.729902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c6e0bbf8-a841-46b6-a2cd-621af950cdff tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1318 microversion: 2.1 time: 0.313721 Oct 04 08:40:30.730491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 434/635] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:30 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1318 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:30.742727 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:30.742727 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:30.742902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:30.743311 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:30.743958 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:30.744905 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:30.745426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:30.745998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:30.810618 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=8 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:30.941156 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:30.956956 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2bad43e0-4a45-4eea-b004-a16a9ebb3d3a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1682 microversion: 2.1 time: 0.224920 Oct 04 08:40:30.958420 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 435/636] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:30 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1682 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:31.284461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-64a004e9-9e3f-4421-85ef-29825d74671b req-acac97c8-f238-4383-9bfb-fd9dc662f12c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "255e6091-63f4-4381-89bc-d6b720ecb68e", "name": "network-vif-deleted", "tag": "24f06c20-7f60-43e3-94eb-e8a55284c173"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:31.294607 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-64a004e9-9e3f-4421-85ef-29825d74671b req-acac97c8-f238-4383-9bfb-fd9dc662f12c service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:31.295146 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-64a004e9-9e3f-4421-85ef-29825d74671b req-acac97c8-f238-4383-9bfb-fd9dc662f12c service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:31.295583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-64a004e9-9e3f-4421-85ef-29825d74671b req-acac97c8-f238-4383-9bfb-fd9dc662f12c service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:31.479013 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-64a004e9-9e3f-4421-85ef-29825d74671b req-acac97c8-f238-4383-9bfb-fd9dc662f12c service nova] Creating event network-vif-deleted:24f06c20-7f60-43e3-94eb-e8a55284c173 for instance 255e6091-63f4-4381-89bc-d6b720ecb68e on hv-cinder-86020 Oct 04 08:40:31.485306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-64a004e9-9e3f-4421-85ef-29825d74671b req-acac97c8-f238-4383-9bfb-fd9dc662f12c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.204453 Oct 04 08:40:31.485926 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 436/637] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:31.759260 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:31.760052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:31.768460 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:31.768953 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:31.769389 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:31.770114 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:31.770581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:31.771015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:31.927854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:31.943235 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a3b0acfd-4391-4163-b58e-e768ef4a6bba tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1318 microversion: 2.1 time: 0.188681 Oct 04 08:40:31.944533 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 437/638] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:31 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1318 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:31.982777 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:31.983406 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:31.990120 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:31.990751 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:31.991390 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:31.992282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:31.992854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:31.993545 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:32.031952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:32.032324 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:32.032754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:32.033206 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 255e6091-63f4-4381-89bc-d6b720ecb68e {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:40:32.092891 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:32.101693 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-12058f60-d1f9-4f6d-af3f-06b06bd9f1ca tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 200 len: 1332 microversion: 2.1 time: 0.123430 Oct 04 08:40:32.102291 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 438/639] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:31 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 1332 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:32.967394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:32.968160 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:32.974854 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:32.975341 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:32.975771 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:32.976474 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:32.976921 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:32.977337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:33.124509 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:33.127099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:33.134774 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-949632f6-a355-40de-a3b8-15d445b4a66e tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1508 microversion: 2.1 time: 0.171760 Oct 04 08:40:33.135413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 439/640] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:32 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1508 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:33.141105 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:33.141487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:33.152426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:33.154312 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:33.154939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:33.155705 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:33.156162 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:33.156590 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:33.180778 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e HTTP/1.1 Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:40:33.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:40:33.184016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': '255e6091-63f4-4381-89bc-d6b720ecb68e'} Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:40:33.185214 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:40:33.186739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:33.188079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:40:33.188079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-07e2234d-39d5-4ba4-84d5-ad106a1d364d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 404 len: 111 microversion: 2.1 time: 0.047560 Oct 04 08:40:33.188079 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 440/641] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:33 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:40:33.192286 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "abb0517b-d386-4284-8b80-20c8248da9c0", "device": "/dev/sdb"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:33.194699 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:33.201801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:33.202283 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:33.202756 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:33.203462 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:33.203910 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:33.204402 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:33.535503 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bc654f36-61d1-4ec9-b8a4-aa8328e46adb tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "POST /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.349476 Oct 04 08:40:33.536284 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 441/642] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:40:33 2022] POST /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments => generated 194 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:40:33.541453 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-4a4468bd-7c2f-4e6b-84b3-945a7e91e40e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '.version_select at 0x7f30a08430d0>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:33.568316 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-4a4468bd-7c2f-4e6b-84b3-945a7e91e40e tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1383308230" status: 202 len: 0 microversion: 2.1 time: 0.030711 Oct 04 08:40:33.568962 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 442/643] 10.222.0.16 () {58 vars in 1221 bytes} [Tue Oct 4 08:40:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1383308230 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:40:34.243114 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:34.243982 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:34.255379 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:34.255939 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:34.256749 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:34.258058 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:34.258959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:34.259775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:34.290072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Going to try to terminate instance {{(pid=127142) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:40:34.391377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e961f1ed-f41c-4275-8cf8-3db585a6a6b5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 204 len: 0 microversion: 2.1 time: 0.155801 Oct 04 08:40:34.392526 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 443/644] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:40:34 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:40:34.400072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-dca0060e-7b05-41a6-9301-e63dbdc87291 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: .version_select at 0x7f30a08d0ee0>, body: {"keypair": {"name": "tempest-keypair-396240972", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDbizRn6krKa5Jn4nx1kTTIzd1+dzVQzQ+9jpgIWLrPifTHjiQamx6HrPQ0BtdKKTY0B0F0/cidkiBrXumbUvV8n7JlkBzq936UMT2i2JPOvQiCM5E64hQ8hHFcBUlM9AQ=="}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:34.416875 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-dca0060e-7b05-41a6-9301-e63dbdc87291 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:40:34.421767 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-dca0060e-7b05-41a6-9301-e63dbdc87291 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'key_pairs'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:40:34.448694 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-dca0060e-7b05-41a6-9301-e63dbdc87291 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.054062 Oct 04 08:40:34.455910 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 444/645] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:40:34 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:40:34.459238 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:34.460637 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:34.471117 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:34.471609 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:34.472036 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:34.472737 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:34.473181 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:34.473599 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:34.682801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:34.699057 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-1b35421a-63ac-4b68-b954-ef6f8c2428ac tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1578 microversion: 2.1 time: 0.247061 Oct 04 08:40:34.699909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 445/646] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:34 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1578 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:35.736878 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:35.737920 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:35.750094 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:35.751173 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:35.752045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:35.753477 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:35.754093 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:35.754624 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:35.928520 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:35.937626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c37bfb97-a05a-45f0-bb18-793137b6e7d8 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1578 microversion: 2.1 time: 0.207266 Oct 04 08:40:35.938083 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 446/647] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:35 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1578 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:36.152292 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1920554411", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd380db5-2756-484c-8401-ea63df26e43d"}], "security_groups": [{"name": "tempest-securitygroup--1138498956"}], "key_name": "tempest-keypair-396240972", "user_data": "}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:36.322345 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] validate_networks() for [('fd380db5-2756-484c-8401-ea63df26e43d', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:40:37.418355 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:40:37.420719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:40:37.420719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:40:37.420719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:40:37.421174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:40:37.430537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:37.431015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:37.431447 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:37.432174 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:37.432618 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:37.433043 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:37.451641 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:40:37.456212 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Getting quotas for user 0c1de2efe2ed42508d7ab4a2e9d91e92 and project 2b940c43c46c402bbc4c3c0461b51061. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:40:37.469844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:40:37.472231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:40:37.536928 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-3b727058-695e-43d0-bc9d-10dff91f6110 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.392101 Oct 04 08:40:37.537385 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 447/648] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:40:36 2022] POST /compute/v2.1/servers => generated 402 bytes in 1394 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 04 08:40:37.541791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:37.542465 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:37.551938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:37.552711 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:37.553163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:37.553894 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:37.554356 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:37.554850 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:37.663345 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:37.680938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-91ddb2d3-e7da-49df-bc74-34f79cf81e86 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1510 microversion: 2.1 time: 0.141520 Oct 04 08:40:37.695109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 448/649] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:37 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1510 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:37.706065 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-755078ca-d6e1-462a-a875-5131e50871ad tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:37.707388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-755078ca-d6e1-462a-a875-5131e50871ad tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:37.806692 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-755078ca-d6e1-462a-a875-5131e50871ad tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:37.819092 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-755078ca-d6e1-462a-a875-5131e50871ad tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1241 microversion: 2.1 time: 0.135242 Oct 04 08:40:37.820383 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 449/650] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:37 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 1241 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:37.983947 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-53643901-4763-42e6-bb7c-58fe27e36f47 req-c06a5d86-7817-4ae6-849c-c2565cf08609 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c1f2b98d-4c4e-47ff-b1d4-19addcee2e27", "name": "network-vif-deleted", "tag": "0e892634-2abc-4cc1-ad5b-83887511e34a"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:37.990826 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:38.001715 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-53643901-4763-42e6-bb7c-58fe27e36f47 req-c06a5d86-7817-4ae6-849c-c2565cf08609 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:38.002744 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-53643901-4763-42e6-bb7c-58fe27e36f47 req-c06a5d86-7817-4ae6-849c-c2565cf08609 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:38.003342 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-53643901-4763-42e6-bb7c-58fe27e36f47 req-c06a5d86-7817-4ae6-849c-c2565cf08609 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:38.018584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-53643901-4763-42e6-bb7c-58fe27e36f47 req-c06a5d86-7817-4ae6-849c-c2565cf08609 service nova] Creating event network-vif-deleted:0e892634-2abc-4cc1-ad5b-83887511e34a for instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 on hv-cinder-86020 Oct 04 08:40:38.026395 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-53643901-4763-42e6-bb7c-58fe27e36f47 req-c06a5d86-7817-4ae6-849c-c2565cf08609 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047199 Oct 04 08:40:38.035322 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 450/651] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:40:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:40:38.712775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:38.714456 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:38.726725 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:38.728824 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:38.728977 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:38.730143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:38.733074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:38.733074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:38.781226 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:38.781712 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:38.782154 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:38.783074 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lazy-loading 'fault' on Instance uuid c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:40:38.836760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ab4b3c2b3f4b4978b0eb472f9c2736b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2f94e3ae140b43739febe1111e18604b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:38.851427 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-25347f62-a7e4-4612-accb-80fdbe857763 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 200 len: 1231 microversion: 2.1 time: 0.144233 Oct 04 08:40:38.852480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 451/652] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:38 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 1231 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:40:38.862413 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:38.864305 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:38.878503 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:38.879267 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:38.880237 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:38.881454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:38.882257 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:38.883032 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:38.977493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:38.987531 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-9594a995-f9aa-4146-9f5b-a6b64113304b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1237 microversion: 2.1 time: 0.133155 Oct 04 08:40:38.988319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 452/653] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:38 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 1237 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:39.878256 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:39.878917 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:39.885642 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:39.886134 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:39.886607 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:39.887367 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:39.889027 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:39.889493 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:39.905537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:40:39.907230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:40:39.909989 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:40:39.911550 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:40:39.913248 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:40:39.915149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:40:39.915149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-81fbfb35-81a7-4f30-84bb-04d33cb408f4 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.034529 Oct 04 08:40:39.915149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 453/654] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:39 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:40:40.014372 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:40:40.015474 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:40:40.022954 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:40.023501 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:40.024152 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:40.024927 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:40:40.025583 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:40:40.026014 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:40:40.091523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:40:40.099366 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-73ee3562-560d-4996-81a7-22e8c0e82235 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1293 microversion: 2.1 time: 0.090098 Oct 04 08:40:40.100029 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 454/655] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:40:40 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 1293 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:40:40.105270 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d1a064e8-21fe-44d0-af70-03444e3a0bbd tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1879105048", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2095819453", "security_groups": [{"name": "tempest-secgroup-smoke-1159980666"}], "block_device_mapping_v2": [{"uuid": "1478f59f-9b4d-4eb0-827a-f36c53d04641", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e71b68ee-781f-4e9d-8cf4-0cfc8be1f31a"}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:40:47.439874 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:47.816957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=12 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:50.095701 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:50.130230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=13 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:00.035542 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:41:00.036109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:41:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:43:38.749366 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:43:38.750086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:43:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:43:49.749907 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:43:49.750928 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:43:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:43:50.137569 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:43:50.138178 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:43:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:43:50.430361 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:43:50.430594 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:43:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:44:20.046454 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:44:20.047131 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:44:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:45:44.120402 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:45:44.120999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:45:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:46:58.768714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:46:58.769381 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:46:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:47:09.763021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:47:09.763742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:47:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:47:10.147446 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:47:10.148288 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:47:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:47:10.439714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:47:10.439946 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:47:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:47:40.056571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:47:40.057501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:47:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:49:04.129700 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:49:04.130682 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:49:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:50:18.789820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:50:18.790540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:50:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:50:29.775577 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:50:29.776383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:50:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:50:30.159361 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:50:30.159867 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:50:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:50:30.449884 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:50:30.450086 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:50:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:51:00.071433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:51:00.072664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:51:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:52:24.142672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:52:24.143262 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:52:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:53:21.503104 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG neutronclient.v2_0.client [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Error message: {"error": {"code": 401, "title": "Unauthorized", "message": "The request you have made requires authentication."}} {{(pid=127143) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Oct 04 08:53:21.503953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 500 len: 0 microversion: 2.1 time: 900.684521 Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Caught error: Not authorized.: nova.exception.Unauthorized: Not authorized. Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack Traceback (most recent call last): Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack exception_handler_v20(status_code, error_body) Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack raise client_exc(message=error_message, Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack neutronclient.common.exceptions.Unauthorized: 401-{'error': {'code': 401, 'title': 'Unauthorized', 'message': 'The request you have made requires authentication.'}} Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack Neutron server returns request_ids: ['req-d1e3ef74-3a3b-4be3-aea1-036555cc2443'] Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack During handling of the above exception, another exception occurred: Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack Traceback (most recent call last): Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/__init__.py", line 95, in __call__ Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return req.get_response(self.application) Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 04 08:53:21.536887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 104, in __call__ Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack self._log_req(req, res, start) Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack self.force_reraise() Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack raise self.value Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/requestlog.py", line 99, in __call__ Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack res = req.get_response(self.application) Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 04 08:53:21.539612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return resp(environ, start_response) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/osprofiler/web.py", line 111, in __call__ Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return request.get_response(self.application) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack response = req.get_response(self._app) Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1313, in send Oct 04 08:53:21.542194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack status, headers, app_iter = self.call_application( Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/request.py", line 1278, in call_application Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack app_iter = application(self.environ, start_response) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return resp(environ, start_response) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return resp(environ, start_response) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/routes/middleware.py", line 153, in __call__ Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack response = self.app(environ, start_response) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 143, in __call__ Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return resp(environ, start_response) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 129, in __call__ Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack resp = self.call_func(req, *args, **kw) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/webob/dec.py", line 193, in call_func Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.func(req, *args, **kwargs) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 486, in __call__ Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self._process_stack(request, action, action_args, Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:21.544742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return method(req=request, **action_args) Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return f(*args, **kwargs) Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return func(*args, **kwargs) Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack [Previous line repeated 10 more times] Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack instances, resv_id = self.compute_api.create( Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self._create_instance( Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ) = self._validate_and_build_base_options( Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 1080, in _validate_and_build_base_options Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack max_network_count = self._check_requested_networks( Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/compute/api.py", line 542, in _check_requested_networks Oct 04 08:53:21.547293 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.network_api.validate_networks(context, requested_networks, Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 2641, in validate_networks Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ports = neutron.list_ports(**params)['ports'] Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 815, in list_ports Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.list('ports', self.ports_path, retrieve_all, Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 372, in list Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack for r in self._pagination(collection, path, **params): Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 387, in _pagination Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack res = self.get(path, params=params) Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.retry_request("GET", action, body=body, Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 04 08:53:21.549992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack return self.do_request(method, action, body=body, Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack ret = obj(*args, **kwargs) Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack self._handle_fault_response(status_code, replybody, resp) Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack File "/opt/stack/nova/nova/network/neutron.py", line 204, in wrapper Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack raise exception.Unauthorized() Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack nova.exception.Unauthorized: Not authorized. Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] http://10.222.0.16/compute/v2.1/servers returned with HTTP 401 Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Returning 401 to user: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:53:21.553743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:21 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/servers (10.222.0.16) Oct 04 08:53:21.555550 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-8bb4f2c8-be4f-4386-9e41-579729848d15 tempest-TestShelveInstance-1264688223 tempest-TestShelveInstance-1264688223-project] Unhandled error: OSError: write error Oct 04 08:53:21.555550 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:21.555550 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:21.555550 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 202/656] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:38:20 2022] POST /compute/v2.1/servers => generated 0 bytes in 900723 msecs (HTTP/1.1 401) 9 headers in 0 bytes (1 switches on core 0) Oct 04 08:53:21.706868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:21.707501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:21.714612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:21.715303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:21.715549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:21.716280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:21.716730 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:21.717147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:21.753679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:21.754149 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:21.754609 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:21.755107 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:21.804757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:21.817055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.273209 Oct 04 08:53:21.818112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:21.818220 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:21 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:21.819244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-bc4b65a4-3cbb-4f71-be25-1cad3ef691b4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:21.819244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:21.819244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:21.819582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 203/657] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:21 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:21.887835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-dbf6e6d2-3b0e-4eaa-8e63-847f8d0b6783 req-26da3d53-0a90-42d5-b419-9d81ff0ee241 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "841d6e59-1b03-4e23-be17-3605627428f1", "tag": "22b85e3a-e060-4164-a7ba-4548ef439545"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:21.898675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-dbf6e6d2-3b0e-4eaa-8e63-847f8d0b6783 req-26da3d53-0a90-42d5-b419-9d81ff0ee241 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:21.899153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-dbf6e6d2-3b0e-4eaa-8e63-847f8d0b6783 req-26da3d53-0a90-42d5-b419-9d81ff0ee241 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:21.899581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-dbf6e6d2-3b0e-4eaa-8e63-847f8d0b6783 req-26da3d53-0a90-42d5-b419-9d81ff0ee241 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:21.912308 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.compute.server_external_events [req-dbf6e6d2-3b0e-4eaa-8e63-847f8d0b6783 req-26da3d53-0a90-42d5-b419-9d81ff0ee241 service nova] Unable to find a host for instance 841d6e59-1b03-4e23-be17-3605627428f1. Dropping event network-changed {{(pid=127143) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 04 08:53:21.913146 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-dbf6e6d2-3b0e-4eaa-8e63-847f8d0b6783 req-26da3d53-0a90-42d5-b419-9d81ff0ee241 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 176 microversion: 2.1 time: 0.092387 Oct 04 08:53:21.913847 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.16) !!! Oct 04 08:53:21.913847 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:21 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.16) Oct 04 08:53:21.914447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [req-dbf6e6d2-3b0e-4eaa-8e63-847f8d0b6783 req-26da3d53-0a90-42d5-b419-9d81ff0ee241 service nova] Unhandled error: OSError: write error Oct 04 08:53:21.914447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:21.914447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:21.914892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 204/658] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:21 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 95 msecs (HTTP/1.1 207) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:22.071736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:22.072616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:22.083761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:22.084645 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:22.085361 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:22.086673 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:22.087637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:22.088417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:22.134616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.218592 Oct 04 08:53:22.135442 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:22.135442 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:22 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:22.136380 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-6a835b0b-ddad-406e-91ad-7579bbb11470 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:22.136380 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:22.136380 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:22.136740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 205/659] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:21 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:22.252154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-553176c5-cd13-432f-8102-b3de5378f85c req-82ef4e47-9292-4a71-b527-90a4b3b9c7a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "841d6e59-1b03-4e23-be17-3605627428f1", "name": "network-vif-plugged", "status": "completed", "tag": "22b85e3a-e060-4164-a7ba-4548ef439545"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:22.262126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-553176c5-cd13-432f-8102-b3de5378f85c req-82ef4e47-9292-4a71-b527-90a4b3b9c7a2 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:22.262638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-553176c5-cd13-432f-8102-b3de5378f85c req-82ef4e47-9292-4a71-b527-90a4b3b9c7a2 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:22.263071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-553176c5-cd13-432f-8102-b3de5378f85c req-82ef4e47-9292-4a71-b527-90a4b3b9c7a2 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:22.275757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.compute.server_external_events [req-553176c5-cd13-432f-8102-b3de5378f85c req-82ef4e47-9292-4a71-b527-90a4b3b9c7a2 service nova] Unable to find a host for instance 841d6e59-1b03-4e23-be17-3605627428f1. Dropping event network-vif-plugged {{(pid=127143) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 04 08:53:22.276650 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-553176c5-cd13-432f-8102-b3de5378f85c req-82ef4e47-9292-4a71-b527-90a4b3b9c7a2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.138954 Oct 04 08:53:22.277182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.16) !!! Oct 04 08:53:22.277182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:22 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.16) Oct 04 08:53:22.277798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [req-553176c5-cd13-432f-8102-b3de5378f85c req-82ef4e47-9292-4a71-b527-90a4b3b9c7a2 service nova] Unhandled error: OSError: write error Oct 04 08:53:22.277798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:22.277798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:22.278153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 206/660] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:22 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 141 msecs (HTTP/1.1 207) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:22.282712 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:22.283198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:22.290817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:22.291343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:22.291839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:22.292564 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:22.293038 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:22.293459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:22.439891 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lazy-loading 'services' on Instance uuid 9a212616-19a8-409e-8dc3-0fe11b28ac76 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:22.632338 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 202 len: 0 microversion: 2.1 time: 0.353098 Oct 04 08:53:22.633470 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:22.633470 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:22 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:22.634162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-f44e17c8-6591-4f7f-b006-2f1add3b678b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:22.634162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:22.634162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:22.634633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 207/661] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:22 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 356 msecs (HTTP/1.1 202) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:22.899725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:22.900404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:22.908321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:22.909256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:22.909812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:22.911668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:22.912417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:22.912946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.065303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'services' on Instance uuid f5ecca0c-50c6-4218-a34e-f4ec70d2686e {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:23.186915 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 202 len: 0 microversion: 2.1 time: 0.550334 Oct 04 08:53:23.188318 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:23.188555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:23.190084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-8cd795b1-9fca-4d81-9ce1-636b42e37a6b tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:23.190084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:23.190084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:23.190837 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 208/662] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:22 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 556 msecs (HTTP/1.1 202) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:23.200887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:23.205633 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:23.218192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.219404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.220391 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.221682 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.222541 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.223504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.259260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.066260 Oct 04 08:53:23.259790 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:23.259987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:23.261618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-f4e0f6f6-a2f5-4aa0-83f6-e059b7d94719 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:23.261618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:23.261618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:23.264029 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 209/663] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:23 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:23.502356 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:23.503105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:23.511705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.512379 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.512825 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.513549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.514006 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.514431 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.529574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:23.531208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:23.533974 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:23.536363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:23.539339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.267340 Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (ip 10.222.0.16) !!! Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (10.222.0.16) Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-6d080eb5-2660-48df-8471-7c4e675c06b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:23.541987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 210/664] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:23 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 270 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:23.543147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:23.543147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:23.546967 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.547468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.547910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.548629 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.549085 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.549506 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.590004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.590727 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.591237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.591775 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:23.666798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:23.679711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.143712 Oct 04 08:53:23.680681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:23.680826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:23.681762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-e6a6d5e5-ec13-4c68-9966-b2a0e280c84b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:23.681762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:23.681762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:23.682015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 211/665] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:23 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:23.690636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:23.691452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:23.702749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.703613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.704384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.705657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.706463 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.707275 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.750215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.066536 Oct 04 08:53:23.750941 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:23.750941 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:23.751797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-99708d49-7b0b-4d7d-a6c8-eda399c17f7d tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:23.751797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:23.751797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:23.752280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 212/666] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:23 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:23.757945 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:23.758446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:23.765124 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.765623 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.766062 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.766842 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.767305 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.767733 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.886235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lazy-loading 'services' on Instance uuid 9a212616-19a8-409e-8dc3-0fe11b28ac76 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:23.967390 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:23.971582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:23.973260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 04 08:53:23.974477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:53:23.975784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.977122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:23.978303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) Oct 04 08:53:23.979823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:23.981059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-13f0bebf-0ed5-44f7-b9d3-105cf32e8999) {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:23.981059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 400 len: 269 microversion: 2.1 time: 0.220787 Oct 04 08:53:23.981059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:23.981059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:23 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:23.981059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-f209624e-7f93-4285-bee1-819c4862917b tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:23.981059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:23.981059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:23.981603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 213/667] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:23 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 223 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:23.981603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:23.981603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:23.988370 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.989523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.990248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:23.991134 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:23.991595 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:23.992015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.090232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'services' on Instance uuid f5ecca0c-50c6-4218-a34e-f4ec70d2686e {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:24.169534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:24.175563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:24.180195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 04 08:53:24.181910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:53:24.184445 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.187420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.188660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) Oct 04 08:53:24.189864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.191159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-609bfdcc-aa73-4355-8828-cecd0df1f170) {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:24.191159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 400 len: 269 microversion: 2.1 time: 0.202556 Oct 04 08:53:24.191159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:24.191159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:24.191159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-dd6bd312-d41a-4e3c-be89-7d7bd1bac734 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:24.191159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.191159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.191680 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 214/668] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:23 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 206 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.191680 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.195179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.209332 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.209980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.210674 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.212037 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.212921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.213726 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.260859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.077230 Oct 04 08:53:24.261405 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:24.261405 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:24.262257 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-d816718a-bb97-4b50-aa43-6e60eaf3836e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:24.262257 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.262257 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.262594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 215/669] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:24 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.267353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.267863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.276409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.276926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.277364 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.278086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.278539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.278993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.302657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:24.304586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.308224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:24.309585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.311910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.042659 Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (ip 10.222.0.16) !!! Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (10.222.0.16) Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-4bf43cb8-cc50-4efd-ad0f-f254a547a4a5 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.313522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 216/670] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:24 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.314068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.314068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.320955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.321528 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.321922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.322701 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.323183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.323610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.362526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.363089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.364666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.365117 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:24.435558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:24.448995 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.140443 Oct 04 08:53:24.449526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:24.449526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:24.450310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-a434c0b8-5719-4681-94c8-cd19a0a87512 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:24.450310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.450310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.450789 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 217/671] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:24 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.455625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.456128 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.465244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.465806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.466348 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.467243 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.467826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.468354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.499429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.047713 Oct 04 08:53:24.499944 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:24.499944 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:24.501099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-233016dd-7585-4a16-adaf-97e48f6f597c tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:24.501099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.501099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.501599 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 218/672] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:24 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.507465 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.507960 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.514294 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.514831 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.515264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.515981 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.516448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.516878 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.607121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:24.609469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:24.612429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:24.613949 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.616782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:24.619481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.108758 Oct 04 08:53:24.619481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:24.619481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:24.619481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-3094d430-1846-4929-9809-d8aa2e8920d9 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:24.619481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.619481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.619481 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 219/673] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:24 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 110 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.620747 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.620747 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.626329 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.626905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.627713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.628453 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.630011 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.630686 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.635527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:24.730663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:24.733758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:24.735236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:24.735890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.737045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:24.737986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.122955 Oct 04 08:53:24.737986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:24.737986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:24.738235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-cd493e1f-e68c-4b0f-809d-3b689114a3af tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:24.738235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.738235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.738492 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 220/674] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:24 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 126 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.749025 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.751716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.758802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.759395 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.759888 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.760662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.761262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.761751 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.796546 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055901 Oct 04 08:53:24.797076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:24.797076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:24.797676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-54a81b2c-2455-4894-9909-e3a26a10cd15 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:24.797676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.797676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.798061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 221/675] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:24 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.803440 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.803940 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.810440 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.810982 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.811427 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.812158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.812605 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.813027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.828061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:24.829707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.831414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:24.832248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:24.834754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.032442 Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (ip 10.222.0.16) !!! Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (10.222.0.16) Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-94517884-aeec-4c1e-b3c5-71e94287e6e0 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.836023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 222/676] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:24 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.837641 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.838253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.844637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.845125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.845560 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.846275 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.846769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.847195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.883905 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.884402 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.884838 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.885298 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:24.934208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:24.943037 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.108554 Oct 04 08:53:24.943637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:24.943745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:24.944249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-676f5433-1c6f-419b-8b83-02e846996fc7 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:24.944249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.944249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.944517 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 223/677] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:24 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.949202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.949709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.956139 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.956641 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.957082 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.960022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:24.960022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:24.960022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:24.985841 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.040297 Oct 04 08:53:24.986260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:24.986260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:24 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:24.986895 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-30ba4992-3f23-449c-af82-c1b68f149f7b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:24.986895 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:24.986895 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:24.987271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 224/678] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:24 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:24.992878 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:24.993379 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:24.999753 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.000249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.000688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.001414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.001870 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.002294 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.088690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:25.091026 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:25.093904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:25.095384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.097815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:25.100575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.105729 Oct 04 08:53:25.100575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:25.100575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:25.100575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-77de70b6-aa7e-42cc-ba47-ff9754d21330 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:25.100575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.100575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.100575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 225/679] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:24 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.103373 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.103873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.110121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.110645 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.111084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.111783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.112237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.112660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.201916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:25.203615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:25.205980 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:25.207216 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.210099 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:25.211676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.107790 Oct 04 08:53:25.211676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:25.211676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:25.211676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-198e4c3d-41cc-4338-842b-44139708c873 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:25.211676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.211676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.211676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 226/680] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:25 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 110 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.212465 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.214134 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.220734 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.221233 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.221677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.222399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.222907 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.223491 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.262225 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.054783 Oct 04 08:53:25.262694 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:25.262694 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:25.263292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-db18ad97-2011-497e-abc7-5894dde5ba10 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:25.263292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.263292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.263589 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 227/681] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:25 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.268382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.268884 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.275743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.276242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.276684 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.277400 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.277859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.278286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.293583 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:25.295212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.297526 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:25.298942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.301201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.032325 Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (ip 10.222.0.16) !!! Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (10.222.0.16) Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-aebd2f32-3dd1-4dc6-bfcf-b873420474b3 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.304522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 228/682] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:25 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.305678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.305678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.310592 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.311078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.311518 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.312232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.312687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.313113 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.348507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.349008 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.349447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.349907 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:25.396897 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:25.404955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.106023 Oct 04 08:53:25.405483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:25.405483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:25.406073 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-5b0055f2-4142-4792-a0f3-4cfa25a8580b tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:25.406073 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.406073 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.406448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 229/683] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:25 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.411138 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.411906 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.418235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.418803 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.419243 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.419962 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.420416 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.420846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.446996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.039661 Oct 04 08:53:25.447545 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:25.447545 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:25.448268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-20d779ad-0f4d-42a9-9cd0-1847ec56a8ce tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:25.448268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.448268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.448634 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 230/684] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:25 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.453839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.454377 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.462161 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.462847 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.463288 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.464008 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.464467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.464895 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.550248 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:25.552561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:25.554868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:25.555665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.556929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:25.557970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.105971 Oct 04 08:53:25.557970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:25.557970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:25.557970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-db8adc43-ae09-4cd3-91d4-990fc92bf825 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:25.557970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.557970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.557970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 231/685] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:25 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.562992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.563483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.571224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.571713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.572150 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.572853 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.573302 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.573728 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.662327 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:25.663875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:25.666182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:25.667511 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.669745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:25.671477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.106249 Oct 04 08:53:25.671477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:25.671477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:25.671477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-da7a99a2-6947-404b-9590-192a93337f03 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:25.671477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.671477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.671477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 232/686] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:25 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.672289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.673323 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.681175 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.681691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.682123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.682863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.683311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.683729 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.720189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052541 Oct 04 08:53:25.720879 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:25.720879 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:25.721644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-b08e74a2-43c1-4a52-acd6-0b404041e5bd tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:25.721644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.721644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.722845 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 233/687] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:25 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.729058 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.729678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.736770 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.737429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.737697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.738415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.739544 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.740250 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.757969 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:25.759717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.761125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:25.761844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:25.763055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.038143 Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (ip 10.222.0.16) !!! Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (10.222.0.16) Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-ce11b439-9886-4213-827a-6d157edfe696 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.764249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 234/688] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:25 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.767453 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.768072 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.775105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.775639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.776084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.776944 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.777404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.777835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.815650 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.816138 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.816575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.817064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:25.860908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:25.871586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.107755 Oct 04 08:53:25.872116 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:25.872116 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:25.872700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-c9b6ea3c-fc70-450f-a62a-c0f5d266b426 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:25.872700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.872700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.873086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 235/689] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:25 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.877398 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-2963793c-2979-46ae-b851-42b9061d9f76 req-415a0620-1ddc-4c11-b65b-6ea9aa4f347b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "841d6e59-1b03-4e23-be17-3605627428f1", "name": "network-vif-deleted", "tag": "22b85e3a-e060-4164-a7ba-4548ef439545"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:25.887709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-2963793c-2979-46ae-b851-42b9061d9f76 req-415a0620-1ddc-4c11-b65b-6ea9aa4f347b service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.888184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-2963793c-2979-46ae-b851-42b9061d9f76 req-415a0620-1ddc-4c11-b65b-6ea9aa4f347b service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.888606 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-2963793c-2979-46ae-b851-42b9061d9f76 req-415a0620-1ddc-4c11-b65b-6ea9aa4f347b service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.906010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.compute.server_external_events [req-2963793c-2979-46ae-b851-42b9061d9f76 req-415a0620-1ddc-4c11-b65b-6ea9aa4f347b service nova] Unable to find a host for instance 841d6e59-1b03-4e23-be17-3605627428f1. Dropping event network-vif-deleted {{(pid=127143) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 04 08:53:25.906953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-2963793c-2979-46ae-b851-42b9061d9f76 req-415a0620-1ddc-4c11-b65b-6ea9aa4f347b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.033165 Oct 04 08:53:25.907508 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.16) !!! Oct 04 08:53:25.907508 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.16) Oct 04 08:53:25.908109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [req-2963793c-2979-46ae-b851-42b9061d9f76 req-415a0620-1ddc-4c11-b65b-6ea9aa4f347b service nova] Unhandled error: OSError: write error Oct 04 08:53:25.908109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.908109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.908467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 236/690] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:25 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 35 msecs (HTTP/1.1 207) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.914132 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.914649 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.921260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.921736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.922179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.923390 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.923854 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.924271 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.951397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.042086 Oct 04 08:53:25.951988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:25.951988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:25 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:25.952772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-65d1e945-05f7-499a-95db-bac8e985d62a tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:25.952772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:25.952772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:25.953126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 237/691] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:25 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:25.959782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:25.960274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:25.966862 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.967345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.967778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:25.969461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:25.969919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:25.970344 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.057104 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:26.058806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:26.061365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:26.063299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.065778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:26.069075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.106376 Oct 04 08:53:26.069075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:26.069075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:26.069075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-50860eb9-9782-4053-a97f-9bd0c8835bb0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:26.069075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.069075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.069075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 238/692] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:25 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.069969 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.069969 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.072917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.073429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.073843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.074578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.075045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.075464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.095813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:26.171819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:26.174819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:26.177793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:26.179089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.181154 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:26.183066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.115438 Oct 04 08:53:26.183066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:26.183066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:26.183066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-1fe8645b-869b-4dba-8ced-f4eb4df9c0bc tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:26.183066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.183066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.183066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 239/693] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:26 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 118 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.188970 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.192669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.199127 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.199617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.200043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.200749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.201194 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.201663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.236493 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.054963 Oct 04 08:53:26.237019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:26.237019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:26.237603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-fc16fac1-a78a-434b-a75b-b898f8ec66c0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:26.237603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.237603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.237855 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 240/694] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:26 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.243163 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.243653 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.250057 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.250727 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.251009 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.251709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.252168 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.252618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.267755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:26.269311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.271198 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:26.272755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.274295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.032303 Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (ip 10.222.0.16) !!! Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (10.222.0.16) Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-ebe8e6d2-ee74-4704-b3b7-739b65bfc94a tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.276819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 241/695] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:26 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.277786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.277786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.286202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.286737 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.287177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.287894 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.288377 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.288772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.326761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.327210 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.327644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.328098 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:26.372125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:26.380930 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.107583 Oct 04 08:53:26.381460 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:26.381460 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:26.382046 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-268475ce-b72a-47ce-ac0e-d32d4703d46f tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:26.382046 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.382046 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.382339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 242/696] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:26 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.387759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.388245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.395147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.395635 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.396074 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.396773 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.397222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.397647 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.425993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.042750 Oct 04 08:53:26.427090 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:26.427090 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:26.427666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-a1afd36e-dfce-4a0b-acbe-d0eeacf2ce3e tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:26.427666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.427666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.427912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 243/697] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:26 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.433140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.433613 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.439781 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.440264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.440694 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.441383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.441827 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.442927 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.539126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:26.542018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:26.543570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:26.544343 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.545661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:26.546772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.115392 Oct 04 08:53:26.546772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:26.546772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:26.546772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-eb1c6f2a-4a1f-45a9-9667-27844faa0dc7 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:26.546772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.546772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.546772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 244/698] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:26 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 118 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.555509 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.556354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.561347 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:26.568720 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.569595 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.570367 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.571729 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.572532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.573269 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.672039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:26.673602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:26.675919 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:26.677818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.679268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:26.680321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.127241 Oct 04 08:53:26.680321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:26.680321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:26.680321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-fd83500d-c368-4e7b-bf62-948aca7b2e0d tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:26.680321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.680321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.680321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 245/699] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:26 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.683048 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.685317 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.691986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.692537 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.693011 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.693836 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.694419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.694966 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.729662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051788 Oct 04 08:53:26.730345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:26.730345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:26.731080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-c4e11f43-6819-427d-8443-87e76dcd642b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:26.731080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.731080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.731470 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 246/700] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:26 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.736575 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.737074 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.744488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.745205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.745906 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.746724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.747219 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.747678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.762507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:26.764093 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.765786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:26.766688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:26.768096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.033272 Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (ip 10.222.0.16) !!! Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 (10.222.0.16) Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-5e59bff4-8b9a-412e-9b13-254d39f53f6f tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.769499 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 247/701] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:26 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 0 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.771931 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.772537 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.778860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.779341 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.779772 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.780465 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.780918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.781337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.821126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.821512 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.821950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.822401 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:26.869798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:26.878627 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.110960 Oct 04 08:53:26.879165 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (ip 10.222.0.16) !!! Oct 04 08:53:26.879165 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 (10.222.0.16) Oct 04 08:53:26.880330 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-fd677f1f-984e-475c-b0a0-d79b85b09592 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Unhandled error: OSError: write error Oct 04 08:53:26.880330 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.880330 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.880792 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 248/702] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:26 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.885126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.885609 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.891940 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.892468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.892932 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.893669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.894161 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.894663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.921218 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.039720 Oct 04 08:53:26.921714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (ip 10.222.0.16) !!! Oct 04 08:53:26.921714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:26 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips (10.222.0.16) Oct 04 08:53:26.922296 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-67b585a5-2931-4ee9-a14e-133a852a6a57 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Unhandled error: OSError: write error Oct 04 08:53:26.922296 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:26.922296 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:26.922689 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 249/703] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:26 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 0 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:26.928124 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:26.928621 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:26.934892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.935394 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.935823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:26.936527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:26.936977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:26.937387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.033359 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:27.034911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:27.037357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:27.038189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.039439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:27.040410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.112963 Oct 04 08:53:27.040410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (ip 10.222.0.16) !!! Oct 04 08:53:27.040410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 (10.222.0.16) Oct 04 08:53:27.040410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-f40b329f-b24e-47df-a73e-11481477246c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Unhandled error: OSError: write error Oct 04 08:53:27.040410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:27.040410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:27.040410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 250/704] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:26 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 0 bytes in 115 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:27.042399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.042947 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.049245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.049744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.050183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.051008 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.052156 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.052673 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.143118 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:27.144669 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:27.146205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:27.147027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.148310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:27.149393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.107902 Oct 04 08:53:27.149393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (ip 10.222.0.16) !!! Oct 04 08:53:27.149393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 (10.222.0.16) Oct 04 08:53:27.149393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-ce08a902-7adc-45d5-997a-c1240b229d01 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Unhandled error: OSError: write error Oct 04 08:53:27.149393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:27.149393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:27.149393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 251/705] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:27 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 0 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:27.152884 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.155088 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.162115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.162676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.163116 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.163851 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.164300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.164717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.198046 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049416 Oct 04 08:53:27.198682 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (ip 10.222.0.16) !!! Oct 04 08:53:27.198682 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Tue Oct 4 08:53:27 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments (10.222.0.16) Oct 04 08:53:27.199345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: CRITICAL nova [None req-2c09b7b6-794d-4a73-8f8e-fcb1ddbc993f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Unhandled error: OSError: write error Oct 04 08:53:27.199345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova OSError: write error Oct 04 08:53:27.199345 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova Oct 04 08:53:27.199736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 252/706] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:27 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 0 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:53:27.204671 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.205211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.211791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.212327 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.212915 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.213808 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.214306 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.214824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.230640 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:27.232211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.234530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:27.235408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.236596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.237764 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:27.237764 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7e967179-f357-4239-8c65-27d4bbcbdd01 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.033043 Oct 04 08:53:27.237764 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 253/707] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:27 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:27.238886 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.239632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.246360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.246876 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.247307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.248012 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.248453 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.248866 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.291054 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.291584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.292058 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.292521 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:27.345245 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:27.353620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-40a9239e-455e-4784-9e6d-52633f2a4ff1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.118405 Oct 04 08:53:27.354208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 254/708] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:27 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 1405 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:27.359353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.359854 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.368534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.369030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.369467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.370185 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.370675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.371109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.398679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cd65bfdd-6e21-4091-9101-50b6f46f93dc tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.043299 Oct 04 08:53:27.399262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 255/709] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:27 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 151 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:53:27.404109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.404618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.411775 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.412382 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.412860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.413574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.414032 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.414457 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.495774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:27.497267 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:27.499664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:27.500869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.502921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:27.505103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b606ca9b-be99-4447-becd-39cad07b7eed tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.098468 Oct 04 08:53:27.505103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 256/710] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:27 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 154 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:27.505103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.505554 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.511789 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.512278 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.512708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.513414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.513868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.514291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.612236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:27.614501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:27.616765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:27.617697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.619111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:27.620179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-91c696c8-8cb5-443e-acd7-6754c6ee4469 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.116087 Oct 04 08:53:27.620179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 257/711] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:27 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 154 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:27.621979 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.624200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.631285 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.631791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.632223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.632954 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.633414 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.633836 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.638107 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=100 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:27.668565 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a38643f8-7039-49ec-921c-53f3027db1ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050555 Oct 04 08:53:27.669152 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 258/712] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:27 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:53:27.673883 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.674379 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.681598 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.682103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.682529 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.683272 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.683731 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.684157 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.701389 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:27.703031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.704908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:27.705823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:27.709549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.712450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:27.712450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-86f5b521-dbe8-4a02-b8e9-02a008808f02 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.034363 Oct 04 08:53:27.712450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 259/713] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:27 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:27.712450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.712450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.718723 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.719214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.719646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.720377 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.720829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.721255 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.757818 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.758480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.758767 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.759225 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:27.801841 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:27.810130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-35078be2-b838-4f04-a113-632c1e5c12ab tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.103533 Oct 04 08:53:27.810755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 260/714] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:27 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 1405 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:27.819700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.820582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.831258 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.832143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.834125 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.835608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.836196 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.836791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.866393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-77af6ae7-a955-4891-b907-0dd06282685b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.054423 Oct 04 08:53:27.866903 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 261/715] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:27 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 151 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:53:27.871903 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.872395 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.880666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.881165 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.881596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.882377 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.882860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.883280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.969948 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:27.971705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:27.973286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:27.974045 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:27.975396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:27.976448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2f59e5f6-acda-4142-8464-285532e552ab tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.105358 Oct 04 08:53:27.976448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 262/716] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:27 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 154 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:27.979088 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:27.979581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:27.986793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.987292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.987787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:27.988733 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:27.989249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:27.990063 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.076730 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:28.078307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:28.080683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:28.081656 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.084111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:28.085232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3335f790-bf85-4a8c-8398-c0fd0a136945 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.104759 Oct 04 08:53:28.085232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 263/717] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:27 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 154 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:28.085724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.087963 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.094294 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.094831 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.095266 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.095987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.096438 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.096859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.131612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-28d89d2f-e4ea-4c5e-b462-1677003bd2cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050238 Oct 04 08:53:28.132183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 264/718] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:28 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:53:28.136928 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.137424 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.146346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.147368 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.148417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.149668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.150494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.151309 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.177859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:28.180978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.183489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:28.184829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.187043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.189215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:28.189215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b457721a-6d4a-4a02-8cf7-bc72776daf6c tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.050135 Oct 04 08:53:28.189215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 265/719] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:28 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:28.193912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.195092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.207617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.209326 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.209783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.210535 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.211020 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.211458 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.250695 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.251183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.251625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.252086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lazy-loading 'fault' on Instance uuid 841d6e59-1b03-4e23-be17-3605627428f1 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:28.297388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0c1de2efe2ed42508d7ab4a2e9d91e92', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b940c43c46c402bbc4c3c0461b51061', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:28.305768 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d7eb572c-a962-43dc-87ec-8e17c534f793 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 200 len: 1405 microversion: 2.1 time: 0.119542 Oct 04 08:53:28.306355 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 266/720] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:28 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 1405 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:28.311572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.312160 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.319118 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.319615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.320051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.320768 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.321221 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.321674 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.347542 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ef22f1e2-93e6-4c8b-8573-6dd468565c44 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips" status: 200 len: 151 microversion: 2.20 time: 0.039922 Oct 04 08:53:28.348678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 267/721] 10.222.0.16 () {60 vars in 1289 bytes} [Tue Oct 4 08:53:28 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210/ips => generated 151 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 04 08:53:28.354516 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.355162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.361948 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.362453 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.362931 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.363705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.364368 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.365204 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.453856 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 HTTP/1.1 Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:28.455396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:28.456890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': '9a212616-19a8-409e-8dc3-0fe11b28ac76', 'id': 'abb0517b-d386-4284-8b80-20c8248da9c0'} Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id abb0517b-d386-4284-8b80-20c8248da9c0. Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:28.458603 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.461103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 is not attached to volume abb0517b-d386-4284-8b80-20c8248da9c0 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:28.463140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1d4db65e-c306-4e8a-ac4e-d66ea83e215c tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0" status: 404 len: 154 microversion: 2.1 time: 0.106512 Oct 04 08:53:28.463140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 268/722] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:28 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76/os-volume_attachments/abb0517b-d386-4284-8b80-20c8248da9c0 => generated 154 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:28.463140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.463654 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.470404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.470989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.471421 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.472115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.472577 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.473755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.560626 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 HTTP/1.1 Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:28.562121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:28.564262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'server_id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e', 'id': '30635ed0-8bd4-45c4-b688-9288e30ee564'} Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 30635ed0-8bd4-45c4-b688-9288e30ee564. Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:28.565209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.567800 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e is not attached to volume 30635ed0-8bd4-45c4-b688-9288e30ee564 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:28.569637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d452dbd6-2d4f-4764-ac61-d292a41ccbc7 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564" status: 404 len: 154 microversion: 2.1 time: 0.104510 Oct 04 08:53:28.569637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 269/723] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:53:28 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments/30635ed0-8bd4-45c4-b688-9288e30ee564 => generated 154 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:28.569637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.571627 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.578035 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.578527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.578994 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.579701 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.580151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.580568 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.616612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-51b8f6f5-e025-4e1d-8611-4cc022b301b8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051419 Oct 04 08:53:28.617184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 270/724] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:28 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:53:28.621920 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.622398 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.629097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.630203 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.631015 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.631756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.632224 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.632662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.650791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:28.652316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.653864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:28.655060 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:28.656459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:28.657840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:28.657840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-906771cb-873b-4487-8cb1-504687e774f2 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.035609 Oct 04 08:53:28.657840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 271/725] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:28 2022] DELETE /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:28.659003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:28.659502 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:28.666237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.666767 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.667204 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.667939 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.668396 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.668817 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.703936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:28.759529 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Doing local delete in cell 4bcedf19-741d-498b-8c07-5371abba3edf(cell1) {{(pid=127143) _delete /opt/stack/nova/nova/compute/api.py:2546}} Oct 04 08:53:28.760289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:28.760709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:28.761135 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:28.762018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: WARNING nova.compute.api [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] instance's host None is down, deleting from database Oct 04 08:53:28.765442 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR oslo.messaging._drivers.impl_rabbit [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [083ca7bc-4f44-4d50-9bf1-fc802cf867cd] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 04 08:53:29.773972 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [083ca7bc-4f44-4d50-9bf1-fc802cf867cd] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 33932. Oct 04 08:53:29.780989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] deallocate_for_instance() {{(pid=127143) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1780}} Oct 04 08:53:29.822842 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Instance cache missing network info. {{(pid=127143) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3300}} Oct 04 08:53:29.833690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Updating instance_info_cache with network_info: [] {{(pid=127143) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Oct 04 08:53:30.276558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:30.360804 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-83911e3d-dfde-410b-93d7-68c499886850 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 204 len: 0 microversion: 2.1 time: 1.704733 Oct 04 08:53:30.362094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 272/726] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:28 2022] DELETE /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 0 bytes in 1707 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:53:30.370065 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:30.370692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:30.378010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.378819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.379263 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.379997 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.380525 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.380953 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.414794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:30.514335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0847c5cd-c682-4f90-9d07-b1cf53983f6f tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 204 len: 0 microversion: 2.20 time: 0.149764 Oct 04 08:53:30.515379 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 273/727] 10.222.0.16 () {60 vars in 1280 bytes} [Tue Oct 4 08:53:30 2022] DELETE /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 04 08:53:30.524846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:30.527918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:30.538128 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.538819 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.539370 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.540220 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.540760 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.541299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.577008 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0b1502c5-e901-4c12-a561-26bdc138112a tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.060162 Oct 04 08:53:30.577675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 274/728] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:30 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments => generated 25 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:53:30.583003 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:30.583679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:30.590036 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.590612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.591172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.591981 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.592488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.592954 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.634749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:30.722419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-61d77ccc-6430-4fc4-91b2-f1c0cd550408 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 204 len: 0 microversion: 2.1 time: 0.143382 Oct 04 08:53:30.723076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 275/729] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:30 2022] DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:53:30.728741 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:30.730022 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] [instance: c1f2b98d-4c4e-47ff-b1d4-19addcee2e27] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:30.737584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.738170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.739474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.740647 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.741213 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.741750 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.761598 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 HTTP/1.1 Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:30.764094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:30.766787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'c1f2b98d-4c4e-47ff-b1d4-19addcee2e27'} Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:30.768190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. Oct 04 08:53:30.770444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.772628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 404 to user: Instance c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:30.772628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-dc904f38-2097-4064-8a3c-65dfb5b1ad07 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27" status: 404 len: 111 microversion: 2.1 time: 0.041833 Oct 04 08:53:30.772628 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 276/730] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:30 2022] GET /compute/v2.1/servers/c1f2b98d-4c4e-47ff-b1d4-19addcee2e27 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:30.774599 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:30.775543 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:30.782671 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.783705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.783946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.785138 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.785611 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.786081 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.804422 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 HTTP/1.1 Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:30.806757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:30.810617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '841d6e59-1b03-4e23-be17-3605627428f1'} Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:30.813325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:30.816692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:30.820030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:30.820030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-68ab4857-6bd0-4962-af12-ea79c92e5ec0 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 404 len: 111 microversion: 2.1 time: 0.040381 Oct 04 08:53:30.820030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 277/731] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:30 2022] DELETE /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:30.820030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:30.820030 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:30.827161 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.827807 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.828351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.829419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.830023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.830615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.864599 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:30.952982 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.compute.api [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Instance is already in deleting state, ignoring this request Oct 04 08:53:30.954091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f8587656-03b2-4443-8727-203a21021b2b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 204 len: 0 microversion: 2.20 time: 0.143358 Oct 04 08:53:30.955230 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 278/732] 10.222.0.16 () {60 vars in 1280 bytes} [Tue Oct 4 08:53:30 2022] DELETE /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 04 08:53:30.965742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:30.968139 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:30.976379 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.976985 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.977497 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:30.978350 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:30.978936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:30.979516 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.016856 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fd2ae3ed-b474-4de3-87df-589037e3ff17 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055768 Oct 04 08:53:31.017438 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 279/733] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:53:30 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e/os-volume_attachments => generated 25 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:53:31.032705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:31.033208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:31.041621 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.042183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.042638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.043497 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.045068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.045529 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.087282 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:31.166783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.compute.api [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Instance is already in deleting state, ignoring this request Oct 04 08:53:31.171140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2f6a3e34-d5a8-4d97-919c-6b54ad31b234 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 204 len: 0 microversion: 2.1 time: 0.152348 Oct 04 08:53:31.172353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 280/734] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:31 2022] DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:53:31.178805 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:31.180112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:31.184423 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=3 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:31.189280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.190151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.190367 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.191410 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.191581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.191916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.225855 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:31.267606 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:UPDATE=16,nova_cell1:DELETE=2,nova_cell1:SELECT=80,nova_cell1:INSERT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:31.339747 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-183a5130-af42-42e6-b6d0-7218b80fb381 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 204 len: 0 microversion: 2.1 time: 0.164349 Oct 04 08:53:31.340524 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 281/735] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:31 2022] DELETE /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:53:31.350635 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:31.351533 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 841d6e59-1b03-4e23-be17-3605627428f1] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:31.359902 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.360581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.361184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.362211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.363005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.364147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.391002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 HTTP/1.1 Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:31.394286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:31.396610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '841d6e59-1b03-4e23-be17-3605627428f1'} Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:31.397921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. Oct 04 08:53:31.402039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:31.404158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 841d6e59-1b03-4e23-be17-3605627428f1 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:31.404158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f42bf099-d3c1-4b27-b516-398e60bd010d tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1" status: 404 len: 111 microversion: 2.1 time: 0.054046 Oct 04 08:53:31.404158 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 282/736] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:31 2022] GET /compute/v2.1/servers/841d6e59-1b03-4e23-be17-3605627428f1 => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:31.412017 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:31.413188 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:31.421119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.421736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.422272 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.423179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.423751 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.424269 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.834758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:31.845228 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:31.846713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:31.858631 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d93eea4f-1fa8-4888-a532-86109275069b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1721 microversion: 2.20 time: 0.448170 Oct 04 08:53:31.858631 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 283/737] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:53:31 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1721 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:53:31.858631 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:31.858631 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:31.861427 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.861913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.862346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:31.863086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:31.864459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:31.865641 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:32.394306 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:32.404380 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-95967af7-e329-457b-82bb-c9e477a69206 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1674 microversion: 2.1 time: 0.553902 Oct 04 08:53:32.404992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 284/738] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:31 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1674 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:32.410111 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:32.411094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:32.418860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:32.419608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:32.420409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:32.421306 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:32.422101 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:32.422715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:32.452579 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:32.552171 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fb8c2478-ea51-4d27-af1c-ec68cd66ac99 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 204 len: 0 microversion: 2.1 time: 0.145493 Oct 04 08:53:32.552753 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 285/739] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:32 2022] DELETE /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:53:32.563000 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:32.563788 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:32.574856 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:32.575354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:32.575898 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:32.576665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:32.577119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:32.577657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:33.306748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:33.323630 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-8101798d-e25a-4716-8a5e-5934ebb3a652 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1514 microversion: 2.1 time: 0.765702 Oct 04 08:53:33.324785 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 286/740] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:32 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1514 bytes in 771 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:33.333865 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fe031273-c538-4e03-b81e-aeed521a3e3c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '.version_select at 0x7f30a075e940>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:33.373952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fe031273-c538-4e03-b81e-aeed521a3e3c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-396240972" status: 202 len: 0 microversion: 2.1 time: 0.046863 Oct 04 08:53:33.374714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 287/741] 10.222.0.16 () {58 vars in 1218 bytes} [Tue Oct 4 08:53:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-396240972 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:53:33.387452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-631d1f22-a9cf-4e53-8215-67a1d2a8e613 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Calling method '.version_select at 0x7f30a076ef70>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:33.427615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-631d1f22-a9cf-4e53-8215-67a1d2a8e613 tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-2095819453" status: 202 len: 0 microversion: 2.1 time: 0.046446 Oct 04 08:53:33.429799 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 288/742] 10.222.0.16 () {58 vars in 1263 bytes} [Tue Oct 4 08:53:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-2095819453 => generated 0 bytes in 54 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:53:33.471084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:33.474129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:33.493029 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:33.495140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:33.497471 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:33.504794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:33.505680 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:33.506489 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.084285 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:34.097545 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-299ff045-f9ab-41bc-a193-2c8e32f13880 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1689 microversion: 2.1 time: 0.663315 Oct 04 08:53:34.111547 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 289/743] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:33 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1689 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:34.118598 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:34.119509 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:34.135381 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.136087 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.136992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.137899 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.138371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.138843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.233679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.234782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.236582 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.237405 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lazy-loading 'fault' on Instance uuid 72e91abf-2688-4fce-a3ea-42775bee6210 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:34.374328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:34.435164 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:34.435164 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '87ca9fc2e013477cad3a3de8e8811274', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e938efbd6804a86946d80ae92e88b16', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:34.437600 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7770f09b-7b31-4078-9624-4eb405004cc3 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 200 len: 1369 microversion: 2.20 time: 0.324054 Oct 04 08:53:34.445076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 290/744] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:53:34 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 1369 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 04 08:53:34.453356 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:34.454662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:34.464694 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.465510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.466346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.467321 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.468044 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.468726 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.831427 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:34.846797 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-82f09f15-6312-4a07-9033-42bf5f22fdd4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1606 microversion: 2.1 time: 0.399380 Oct 04 08:53:34.849705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 291/745] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:34 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1606 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:34.856385 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-c41d6588-6c40-4be4-a387-74411632b19c req-956138d7-2fd2-4cbd-9438-0f1d9c319656 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "72e91abf-2688-4fce-a3ea-42775bee6210", "name": "network-vif-deleted", "tag": "8c389990-706b-42b4-8cba-e3f6d810fbb6"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:34.870863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-c41d6588-6c40-4be4-a387-74411632b19c req-956138d7-2fd2-4cbd-9438-0f1d9c319656 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.871837 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-c41d6588-6c40-4be4-a387-74411632b19c req-956138d7-2fd2-4cbd-9438-0f1d9c319656 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.872447 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-c41d6588-6c40-4be4-a387-74411632b19c req-956138d7-2fd2-4cbd-9438-0f1d9c319656 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.907062 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-c41d6588-6c40-4be4-a387-74411632b19c req-956138d7-2fd2-4cbd-9438-0f1d9c319656 service nova] Creating event network-vif-deleted:8c389990-706b-42b4-8cba-e3f6d810fbb6 for instance 72e91abf-2688-4fce-a3ea-42775bee6210 on hv-cinder-86020 Oct 04 08:53:34.917713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-c41d6588-6c40-4be4-a387-74411632b19c req-956138d7-2fd2-4cbd-9438-0f1d9c319656 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065779 Oct 04 08:53:34.928858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 292/746] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:53:34.940699 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:34.942890 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:34.967667 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.969887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.970839 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:34.979885 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:34.982195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:34.985884 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:35.150690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:35.152902 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:35.153571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:35.155662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lazy-loading 'fault' on Instance uuid 9a212616-19a8-409e-8dc3-0fe11b28ac76 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:35.393183 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c42d5aec138420a90b87226dbe6dc20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0b762e9e807416b925d8d34bb16fbf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:35.406309 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2bb829f1-7a7c-49bd-a351-bed53d96caf0 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 200 len: 1308 microversion: 2.1 time: 0.475518 Oct 04 08:53:35.407405 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 293/747] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:34 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 1308 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:35.418010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:35.419069 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:35.435129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:35.436209 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:35.438084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:35.443479 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:35.444715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:35.445341 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:35.510289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:35.511144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:35.511788 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:35.512411 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'fault' on Instance uuid f5ecca0c-50c6-4218-a34e-f4ec70d2686e {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:35.644075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:35.671573 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fe5f88af-b3ee-4700-890b-4230768018fe tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 200 len: 1336 microversion: 2.1 time: 0.262567 Oct 04 08:53:35.675731 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 294/748] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:35 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 1336 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:35.936600 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-59a3c137-659c-4d30-8b13-e229ef7503f1 req-36ed50fe-d54b-497a-b893-e9bc16244d69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5ecca0c-50c6-4218-a34e-f4ec70d2686e", "name": "network-vif-deleted", "tag": "84c7f948-82c8-4e44-86f1-d8ace0e90f47"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:35.982595 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-59a3c137-659c-4d30-8b13-e229ef7503f1 req-36ed50fe-d54b-497a-b893-e9bc16244d69 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:35.983281 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-59a3c137-659c-4d30-8b13-e229ef7503f1 req-36ed50fe-d54b-497a-b893-e9bc16244d69 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:35.983741 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-59a3c137-659c-4d30-8b13-e229ef7503f1 req-36ed50fe-d54b-497a-b893-e9bc16244d69 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.003455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-59a3c137-659c-4d30-8b13-e229ef7503f1 req-36ed50fe-d54b-497a-b893-e9bc16244d69 service nova] Creating event network-vif-deleted:84c7f948-82c8-4e44-86f1-d8ace0e90f47 for instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e on hv-cinder-86020 Oct 04 08:53:36.012281 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-59a3c137-659c-4d30-8b13-e229ef7503f1 req-36ed50fe-d54b-497a-b893-e9bc16244d69 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.332134 Oct 04 08:53:36.013779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 295/749] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:53:36.024288 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:36.026295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:36.039054 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.041031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.041881 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.043042 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.043616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.044177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.078861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] HTTP exception thrown: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Request method failure captured: Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 HTTP/1.1 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BE8CUPPt8UTo8wxyjiitahg4WwA74AGkiBCEnsdRaJLKQNG2l38LIokg_eUW1qaBKdMxDHY2oHljwCLhRSfHbTrbeIwxfV0dPgvGFHeTebAxT7VFsODm3KBGGbp9WSsMikMmG2HpFYxWvFVsRLrymAAwp5gI4QbbJxZDYhCBs0Ph0 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Openstack-Nova-Api-Version: 2.20 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:36.081912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e938efbd6804a86946d80ae92e88b16", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 87ca9fc2e013477cad3a3de8e8811274 Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:36.084078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '72e91abf-2688-4fce-a3ea-42775bee6210'} Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:36.084813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:36.086096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.088434 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Returning 404 to user: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:36.088434 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-84b26b8e-f040-423a-94b4-5513699a93c6 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 404 len: 111 microversion: 2.20 time: 0.069998 Oct 04 08:53:36.088434 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 296/750] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:53:36 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:53:36.094351 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:36.095725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:36.108167 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.109068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.110215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.111310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.111873 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.112472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.203792 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.204701 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.205560 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.206359 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lazy-loading 'fault' on Instance uuid 9606a52d-3c5d-4287-afb7-c84f5e245c9a {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:36.600598 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:36.626333 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9f24a6f8-7f95-4bde-88bd-ea9abe3a3e32 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 200 len: 1327 microversion: 2.1 time: 0.536669 Oct 04 08:53:36.642507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 297/751] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:36 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 1327 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:36.657758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1ec2bfc3-9af6-4bb4-ba7b-189862d5bf85 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '.version_select at 0x7f30a075e550>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:36.705290 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1ec2bfc3-9af6-4bb4-ba7b-189862d5bf85 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-596417858" status: 204 len: 0 microversion: 2.20 time: 0.060528 Oct 04 08:53:36.706744 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 298/752] 10.222.0.16 () {60 vars in 1259 bytes} [Tue Oct 4 08:53:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-596417858 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 04 08:53:36.731948 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:36.733407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] [instance: 9a212616-19a8-409e-8dc3-0fe11b28ac76] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:36.768505 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.769376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.770807 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.772159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.773120 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.774141 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.807814 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] HTTP exception thrown: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 could not be found. Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Request method failure captured: Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 HTTP/1.1 Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_En9AzoFPHOnKQY0G-Mm4FwwzD_8FBRGpuLUfdpsnHPinQPFmW7-9rAyJb3FDtiPl5_L5W5or7bYAxGHS_S-VAB-lKW-arZWMR5b57RH96UYkpls3LT9O2d05HPumWmSo3wIjNia1Lq6lriXRv_KoqVTzMMd-P0EkpM3SS3XzvxF_s Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:36.815679 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f0b762e9e807416b925d8d34bb16fbf0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f0b762e9e807416b925d8d34bb16fbf0", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: f0b762e9e807416b925d8d34bb16fbf0 Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-663201796 Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 9c42d5aec138420a90b87226dbe6dc20 Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesSnapshotTestJSON-663201796-project Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 could not be found. Oct 04 08:53:36.817968 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '9a212616-19a8-409e-8dc3-0fe11b28ac76'} Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 could not be found. Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:36.818779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 could not be found. Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 could not be found. Oct 04 08:53:36.819951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.821075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] Returning 404 to user: Instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:36.838764 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-621f9b1c-94b0-4fee-8751-d85056977868 tempest-VolumesSnapshotTestJSON-663201796 tempest-VolumesSnapshotTestJSON-663201796-project] 10.222.0.16 "GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76" status: 404 len: 111 microversion: 2.1 time: 0.097040 Oct 04 08:53:36.839987 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 299/753] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:36 2022] GET /compute/v2.1/servers/9a212616-19a8-409e-8dc3-0fe11b28ac76 => generated 111 bytes in 132 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:36.866255 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:36.886835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f5ecca0c-50c6-4218-a34e-f4ec70d2686e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:36.916177 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.920708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.921506 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.922530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:36.923858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:36.924748 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:36.957983 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e could not be found. Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e HTTP/1.1 Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:36.962270 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e could not be found. Oct 04 08:53:36.963708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'f5ecca0c-50c6-4218-a34e-f4ec70d2686e'} Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e could not be found. Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:36.964446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e could not be found. Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e could not be found. Oct 04 08:53:36.965725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:36.970289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f5ecca0c-50c6-4218-a34e-f4ec70d2686e could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:36.971709 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9f6d8050-e6fe-49b0-bd76-592407e88cd5 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e" status: 404 len: 111 microversion: 2.1 time: 0.126790 Oct 04 08:53:36.973574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 300/754] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:36 2022] GET /compute/v2.1/servers/f5ecca0c-50c6-4218-a34e-f4ec70d2686e => generated 111 bytes in 133 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:36.994928 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-06c29b2b-c064-4a76-bc95-8753ef7b02c9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '.version_select at 0x7f30a07220d0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:37.064288 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-06c29b2b-c064-4a76-bc95-8753ef7b02c9 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2108872820" status: 202 len: 0 microversion: 2.1 time: 0.082753 Oct 04 08:53:37.065144 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 301/755] 10.222.0.16 () {58 vars in 1221 bytes} [Tue Oct 4 08:53:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2108872820 => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:53:37.082162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-afe5325a-9b54-4347-a320-5eacd15d79aa req-06233a5c-377a-431c-afc5-eca83bf25c70 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a212616-19a8-409e-8dc3-0fe11b28ac76", "name": "network-vif-deleted", "tag": "f7276a7a-f322-4f2a-94f2-80a317bcf537"}, {"server_uuid": "9606a52d-3c5d-4287-afb7-c84f5e245c9a", "name": "network-vif-deleted", "tag": "321b30d3-511f-4d46-a3a8-f9853c32c5d9"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:37.117031 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-afe5325a-9b54-4347-a320-5eacd15d79aa req-06233a5c-377a-431c-afc5-eca83bf25c70 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:37.117580 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-afe5325a-9b54-4347-a320-5eacd15d79aa req-06233a5c-377a-431c-afc5-eca83bf25c70 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:37.118147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-afe5325a-9b54-4347-a320-5eacd15d79aa req-06233a5c-377a-431c-afc5-eca83bf25c70 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:37.155282 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-afe5325a-9b54-4347-a320-5eacd15d79aa req-06233a5c-377a-431c-afc5-eca83bf25c70 service nova] Creating event network-vif-deleted:f7276a7a-f322-4f2a-94f2-80a317bcf537 for instance 9a212616-19a8-409e-8dc3-0fe11b28ac76 on hv-cinder-86020 Oct 04 08:53:37.155823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-afe5325a-9b54-4347-a320-5eacd15d79aa req-06233a5c-377a-431c-afc5-eca83bf25c70 service nova] Creating event network-vif-deleted:321b30d3-511f-4d46-a3a8-f9853c32c5d9 for instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a on hv-cinder-86020 Oct 04 08:53:37.182443 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-afe5325a-9b54-4347-a320-5eacd15d79aa req-06233a5c-377a-431c-afc5-eca83bf25c70 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.112275 Oct 04 08:53:37.194218 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 302/756] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:37 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:53:37.666113 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:37.680200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:37.723135 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:37.727440 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:37.728530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:37.731126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:37.735346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:37.742527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:37.793740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] HTTP exception thrown: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Request method failure captured: Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a HTTP/1.1 Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_C2_30scAmridThWKmbW4chXmDOUW8CmDpNKeNqJ2RQzuBAKGAhffCKe5-lQA3FxfABPIN2eVqtWUyGUBPonQ0fz8Zk_vT-9Jddfc_ED9G8U327X1gi5nv7wGesdYWJ3Z1saIJjI7TcpD9slv5w1XHI6vZ0o5eIoa4bIW0dwpla5OQ Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:37.799910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5d21789971964770bdf417547cd19840"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5d21789971964770bdf417547cd19840", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 78634b34107d49cdbf0126014bf8f6f5 Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:53:37.802061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '9606a52d-3c5d-4287-afb7-c84f5e245c9a'} Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:37.803586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:53:37.806254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:37.809976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Returning 404 to user: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:37.811399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-516e355b-9a95-4091-87a6-8087e4a53bef tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 404 len: 111 microversion: 2.1 time: 0.155866 Oct 04 08:53:37.816179 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 303/757] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:37 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 111 bytes in 162 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:37.822352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7cb8c7f6-9168-4593-b18f-04149760708f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '.version_select at 0x7f30a06e0820>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:37.867537 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7cb8c7f6-9168-4593-b18f-04149760708f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2045943656" status: 202 len: 0 microversion: 2.1 time: 0.049499 Oct 04 08:53:37.872683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 304/758] 10.222.0.16 () {58 vars in 1221 bytes} [Tue Oct 4 08:53:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2045943656 => generated 0 bytes in 55 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:53:38.294523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:38.295740 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:38.315739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.316762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.317455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.318455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.319203 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.320178 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.324713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:38.343956 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e HTTP/1.1 Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:38.346698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.348850 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '255e6091-63f4-4381-89bc-d6b720ecb68e'} Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:38.349619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.351840 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.356572 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:38.357859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-283b1848-0437-4fb9-930e-b73b58ff844a tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 404 len: 111 microversion: 2.1 time: 0.067521 Oct 04 08:53:38.362097 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 305/759] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:38 2022] DELETE /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:53:38.369860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:38.370948 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 255e6091-63f4-4381-89bc-d6b720ecb68e] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:38.378370 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.380743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.381555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.382610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.383567 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.384252 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.404978 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e HTTP/1.1 Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:38.407639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.409021 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '255e6091-63f4-4381-89bc-d6b720ecb68e'} Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:38.409759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. Oct 04 08:53:38.411562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.414830 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 255e6091-63f4-4381-89bc-d6b720ecb68e could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:38.416071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-09c335a7-57aa-4d2a-aa56-1ea9ce61a3f1 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e" status: 404 len: 111 microversion: 2.1 time: 0.051954 Oct 04 08:53:38.420779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 306/760] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:38 2022] GET /compute/v2.1/servers/255e6091-63f4-4381-89bc-d6b720ecb68e => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:38.427794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:38.428749 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:38.454142 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.455064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.455884 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.458475 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.458786 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.459527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.528804 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c HTTP/1.1 Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:38.531863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.533510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '6dee148f-e8b5-437c-9d80-4d784091314c'} Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:38.534259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.536187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.537531 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:38.537531 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-de3a0dbb-c7b8-48ca-86ab-4450686d01c2 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 404 len: 111 microversion: 2.1 time: 0.111822 Oct 04 08:53:38.537531 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 307/761] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:38 2022] DELETE /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 111 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:38.556501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:38.559055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: 6dee148f-e8b5-437c-9d80-4d784091314c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:38.584432 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.585249 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.585877 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.587034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:38.587845 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:38.588471 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:38.608419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c HTTP/1.1 Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:38.610530 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.612328 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '6dee148f-e8b5-437c-9d80-4d784091314c'} Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:38.613262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. Oct 04 08:53:38.617383 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:38.626636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance 6dee148f-e8b5-437c-9d80-4d784091314c could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:38.626636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fdfbd867-3089-4026-99e9-a72a4cc6391c tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c" status: 404 len: 111 microversion: 2.1 time: 0.085112 Oct 04 08:53:38.626636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 308/762] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:38 2022] GET /compute/v2.1/servers/6dee148f-e8b5-437c-9d80-4d784091314c => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:39.237264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:39.238086 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:39.251386 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:39.252814 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:39.253903 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:39.255529 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:39.256352 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:39.257390 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:39.286042 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] HTTP exception thrown: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Request method failure captured: Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 HTTP/1.1 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BE8CUPPt8UTo8wxyjiitahg4WwA74AGkiBCEnsdRaJLKQNG2l38LIokg_eUW1qaBKdMxDHY2oHljwCLhRSfHbTrbeIwxfV0dPgvGFHeTebAxT7VFsODm3KBGGbp9WSsMikMmG2HpFYxWvFVsRLrymAAwp5gI4QbbJxZDYhCBs0Ph0 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Openstack-Nova-Api-Version: 2.20 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:39.290835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e938efbd6804a86946d80ae92e88b16", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 87ca9fc2e013477cad3a3de8e8811274 Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.293607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '72e91abf-2688-4fce-a3ea-42775bee6210'} Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:39.296056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.299206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.305605 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Returning 404 to user: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:39.307421 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5a853f55-705e-4485-8411-4b3d5067fd1b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 404 len: 111 microversion: 2.20 time: 0.087319 Oct 04 08:53:39.313497 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 309/763] 10.222.0.16 () {60 vars in 1280 bytes} [Tue Oct 4 08:53:39 2022] DELETE /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 111 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 04 08:53:39.323175 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:39.324411 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 72e91abf-2688-4fce-a3ea-42775bee6210] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:39.335726 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:39.336615 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:39.337494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:39.338964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:39.339907 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:39.340761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:39.370451 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] HTTP exception thrown: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Request method failure captured: Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 HTTP/1.1 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BE8CUPPt8UTo8wxyjiitahg4WwA74AGkiBCEnsdRaJLKQNG2l38LIokg_eUW1qaBKdMxDHY2oHljwCLhRSfHbTrbeIwxfV0dPgvGFHeTebAxT7VFsODm3KBGGbp9WSsMikMmG2HpFYxWvFVsRLrymAAwp5gI4QbbJxZDYhCBs0Ph0 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Openstack-Nova-Api-Version: 2.20 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:39.373668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e938efbd6804a86946d80ae92e88b16", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 87ca9fc2e013477cad3a3de8e8811274 Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.375698 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '72e91abf-2688-4fce-a3ea-42775bee6210'} Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:39.376918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. Oct 04 08:53:39.378397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.381869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Returning 404 to user: Instance 72e91abf-2688-4fce-a3ea-42775bee6210 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:39.381869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ec117677-0575-4433-987f-449f945cd919 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210" status: 404 len: 111 microversion: 2.20 time: 0.060539 Oct 04 08:53:39.382461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 310/764] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:53:39 2022] GET /compute/v2.1/servers/72e91abf-2688-4fce-a3ea-42775bee6210 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:53:39.876611 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:39.878290 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:39.893979 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:39.895472 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:39.896697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:39.898378 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:39.899597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:39.900964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:39.935182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 HTTP/1.1 Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:39.938926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:39.942779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'a8351183-2123-48d0-9f68-c06ab2829d71'} Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:39.944214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:39.946485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:39.952765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:39.954127 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-139964cb-198e-478f-ac74-713f42a7d9c6 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 404 len: 111 microversion: 2.1 time: 0.087940 Oct 04 08:53:39.958964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 311/765] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:39 2022] DELETE /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:39.963729 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-d3900a56-6686-4628-b9b0-0cb62ac5d727 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'create', calling method: .version_select at 0x7f30a0722c10>, body: {"keypair": {"name": "tempest-keypair-35649159", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOVWyDWzj/zOcBp8L2pc+PEM8CgUFc3zfjWqAmSzuoX5bMvcx2LfXpVWoFhNK0Rf8UCmWqd0LGJu2AJJJZZnnyAslW9k0xZNJ8h0g1+VqCQX/XQvMtHYUJCF/xFVywXO8g=="}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:39.986240 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-d3900a56-6686-4628-b9b0-0cb62ac5d727 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for project 5d21789971964770bdf417547cd19840. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:53:39.991537 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-d3900a56-6686-4628-b9b0-0cb62ac5d727 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for user 78634b34107d49cdbf0126014bf8f6f5 and project 5d21789971964770bdf417547cd19840. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:53:40.003573 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=2,nova_api:DELETE=5 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:40.026367 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-d3900a56-6686-4628-b9b0-0cb62ac5d727 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.069515 Oct 04 08:53:40.027929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 312/766] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:53:39 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:53:40.037723 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:40.039429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] [instance: a8351183-2123-48d0-9f68-c06ab2829d71] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:40.051838 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:40.052946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:40.054071 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:40.055889 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:40.056559 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:40.057495 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:40.087494 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] HTTP exception thrown: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Request method failure captured: Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 HTTP/1.1 Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BNhM__dEmjequXs73xrGGbBueH9z7bM2Nsr_ulOp4UmSh7V1sdc6GHRaEcLkM_lDUcxz0-3HxGx5Xx7M_RG3btReF-C3C5o15ucYr2Vmvf4gQeoqqq_uudbpYN8b36r1oS2FIt9FZlFxOLP0s17rXoRok-mjZdEMnZ7dqvM859SNk Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:40.090913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2b940c43c46c402bbc4c3c0461b51061"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2b940c43c46c402bbc4c3c0461b51061", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 2b940c43c46c402bbc4c3c0461b51061 Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-231091262 Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 0c1de2efe2ed42508d7ab4a2e9d91e92 Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeNegativeTest-231091262-project Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:40.093957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'a8351183-2123-48d0-9f68-c06ab2829d71'} Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:40.095681 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. Oct 04 08:53:40.098170 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:40.105023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] Returning 404 to user: Instance a8351183-2123-48d0-9f68-c06ab2829d71 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:40.105023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-522bd746-4c47-43a0-a6da-6f7c24a02db4 tempest-AttachVolumeNegativeTest-231091262 tempest-AttachVolumeNegativeTest-231091262-project] 10.222.0.16 "GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71" status: 404 len: 111 microversion: 2.1 time: 0.067645 Oct 04 08:53:40.107084 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 313/767] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:40 2022] GET /compute/v2.1/servers/a8351183-2123-48d0-9f68-c06ab2829d71 => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:42.982222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:42.983844 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:43.002619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:43.003230 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:43.003913 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:43.009893 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:43.013333 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:43.013801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:43.092892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] HTTP exception thrown: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Request method failure captured: Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 HTTP/1.1 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BE8CUPPt8UTo8wxyjiitahg4WwA74AGkiBCEnsdRaJLKQNG2l38LIokg_eUW1qaBKdMxDHY2oHljwCLhRSfHbTrbeIwxfV0dPgvGFHeTebAxT7VFsODm3KBGGbp9WSsMikMmG2HpFYxWvFVsRLrymAAwp5gI4QbbJxZDYhCBs0Ph0 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Openstack-Nova-Api-Version: 2.20 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:43.101636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e938efbd6804a86946d80ae92e88b16", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 87ca9fc2e013477cad3a3de8e8811274 Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.103558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '54819895-e219-4341-8f1e-5543b7357315'} Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:43.104433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.106468 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.113983 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Returning 404 to user: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:43.115676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f7f03c89-75b5-4e65-b2da-1ba2fb8f0883 tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "DELETE /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 404 len: 111 microversion: 2.20 time: 0.142006 Oct 04 08:53:43.116477 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 314/768] 10.222.0.16 () {60 vars in 1280 bytes} [Tue Oct 4 08:53:42 2022] DELETE /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 111 bytes in 144 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:53:43.162391 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:43.162391 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] [instance: 54819895-e219-4341-8f1e-5543b7357315] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:43.196220 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:43.196957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:43.198062 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:43.200365 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:43.201642 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:43.202452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:43.227512 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] HTTP exception thrown: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Request method failure captured: Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 HTTP/1.1 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BE8CUPPt8UTo8wxyjiitahg4WwA74AGkiBCEnsdRaJLKQNG2l38LIokg_eUW1qaBKdMxDHY2oHljwCLhRSfHbTrbeIwxfV0dPgvGFHeTebAxT7VFsODm3KBGGbp9WSsMikMmG2HpFYxWvFVsRLrymAAwp5gI4QbbJxZDYhCBs0Ph0 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Openstack-Nova-Api-Version: 2.20 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:43.230200 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e938efbd6804a86946d80ae92e88b16"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e938efbd6804a86946d80ae92e88b16", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 3e938efbd6804a86946d80ae92e88b16 Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1787866423 Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 87ca9fc2e013477cad3a3de8e8811274 Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1787866423-project Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.232034 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '54819895-e219-4341-8f1e-5543b7357315'} Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:43.232910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. Oct 04 08:53:43.234480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:43.236784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] Returning 404 to user: Instance 54819895-e219-4341-8f1e-5543b7357315 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:43.237833 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a94553ec-2c6a-40eb-a2be-16a5b3a9e20b tempest-AttachVolumeShelveTestJSON-1787866423 tempest-AttachVolumeShelveTestJSON-1787866423-project] 10.222.0.16 "GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315" status: 404 len: 111 microversion: 2.20 time: 0.116167 Oct 04 08:53:43.244578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 315/769] 10.222.0.16 () {60 vars in 1277 bytes} [Tue Oct 4 08:53:43 2022] GET /compute/v2.1/servers/54819895-e219-4341-8f1e-5543b7357315 => generated 111 bytes in 121 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:53:44.411330 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:44.413789 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:44.436769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:44.436769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:44.436769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:44.436769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:44.437507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:44.437675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:44.494662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:44.699251 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5226109e-e658-453d-9458-9dd83458abd1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 204 len: 0 microversion: 2.1 time: 0.295247 Oct 04 08:53:44.701606 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 316/770] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:44 2022] DELETE /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 0 bytes in 299 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:53:44.724496 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:44.725763 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:44.738925 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:44.740087 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:44.740863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:44.743662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:44.744754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:44.745694 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:45.777670 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:45.786105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f28ec445-6158-4bf8-8e76-83bde7aa4153 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1507 microversion: 2.1 time: 1.076041 Oct 04 08:53:45.802761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 317/771] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:44 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1507 bytes in 1095 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:45.809991 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1741386597", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd08aa17-8a80-4dac-a656-3195d8433ec8"}], "security_groups": [{"name": "tempest-securitygroup--1323726239"}], "key_name": "tempest-keypair-35649159", "user_data": "}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:46.117237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] validate_networks() for [('fd08aa17-8a80-4dac-a656-3195d8433ec8', None, None, None, None, None)] {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:53:48.324311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:53:48.326476 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:53:48.327409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:53:48.328128 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:53:48.328875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:53:48.345282 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:48.345799 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:48.346360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:48.347539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:48.348608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:48.349362 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:48.389756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for project 5d21789971964770bdf417547cd19840. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:53:48.398855 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Getting quotas for user 78634b34107d49cdbf0126014bf8f6f5 and project 5d21789971964770bdf417547cd19840. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:53:48.423625 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:53:48.428041 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:53:48.525518 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR oslo.messaging._drivers.impl_rabbit [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [d073aa43-7f0c-4dae-a99b-4dd2338fbd5d] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 04 08:53:49.541963 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [d073aa43-7f0c-4dae-a99b-4dd2338fbd5d] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 34222. Oct 04 08:53:49.569574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-53b3c9c0-2b53-47a8-8f6a-0b71792f7b00 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.763129 Oct 04 08:53:49.572094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 318/772] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:53:45 2022] POST /compute/v2.1/servers => generated 402 bytes in 3768 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:53:49.596315 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:49.596962 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:49.627397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:49.629043 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:49.629981 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:49.631759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:49.632767 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:49.633664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:49.701181 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:49.702423 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:49.703779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:49.704388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'fault' on Instance uuid b9fd4027-9ae5-4ab2-afc6-c7eb723204cf {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:49.788474 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:49.808311 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-315a6a62-2f26-421e-b486-024099843128 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1312 microversion: 2.1 time: 0.232829 Oct 04 08:53:49.811929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 319/773] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:49 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1312 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:49.817724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-a3ff9156-f7ba-44df-88ba-0dfa26964013 req-ab51e389-b9a2-45bd-8c00-4235d7ecb910 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9fd4027-9ae5-4ab2-afc6-c7eb723204cf", "name": "network-vif-deleted", "tag": "364e2511-1153-48e7-8595-032dbdba4297"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:49.842995 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-a3ff9156-f7ba-44df-88ba-0dfa26964013 req-ab51e389-b9a2-45bd-8c00-4235d7ecb910 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:49.846831 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-a3ff9156-f7ba-44df-88ba-0dfa26964013 req-ab51e389-b9a2-45bd-8c00-4235d7ecb910 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:49.847595 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-a3ff9156-f7ba-44df-88ba-0dfa26964013 req-ab51e389-b9a2-45bd-8c00-4235d7ecb910 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:49.885737 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-a3ff9156-f7ba-44df-88ba-0dfa26964013 req-ab51e389-b9a2-45bd-8c00-4235d7ecb910 service nova] Creating event network-vif-deleted:364e2511-1153-48e7-8595-032dbdba4297 for instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf on hv-cinder-86020 Oct 04 08:53:49.895454 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-a3ff9156-f7ba-44df-88ba-0dfa26964013 req-ab51e389-b9a2-45bd-8c00-4235d7ecb910 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080880 Oct 04 08:53:49.896717 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 320/774] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:53:49.905783 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-16614a92-1d62-47d2-8b45-203f721feac9 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:49.906828 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-16614a92-1d62-47d2-8b45-203f721feac9 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:50.111704 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-16614a92-1d62-47d2-8b45-203f721feac9 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:50.134651 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-16614a92-1d62-47d2-8b45-203f721feac9 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1236 microversion: 2.1 time: 0.235599 Oct 04 08:53:50.135627 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 321/775] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:49 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1236 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:50.170801 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:53:50.171403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:53:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:53:50.459072 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:53:50.459275 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:53:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:53:50.848892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:50.850581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:50.862211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:50.863757 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:50.864936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:50.866371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:50.869513 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:50.870279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:50.961545 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:50.962150 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:50.962655 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:50.963162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'fault' on Instance uuid b9fd4027-9ae5-4ab2-afc6-c7eb723204cf {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:51.053424 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:51.083831 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a4053177-19e5-49f0-8051-1466f53e44e8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 200 len: 1312 microversion: 2.1 time: 0.241129 Oct 04 08:53:51.084690 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 322/776] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:50 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 1312 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:51.168754 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:51.170791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:51.190518 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:51.191585 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:51.192484 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:51.193848 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:51.194878 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:51.195846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:51.400425 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:51.446028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f03dc9c3-e39e-4f4b-aac1-9f631e573c1c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1240 microversion: 2.1 time: 0.287317 Oct 04 08:53:51.446674 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 323/777] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:51 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1240 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:53:52.125143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:52.126174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: b9fd4027-9ae5-4ab2-afc6-c7eb723204cf] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:52.138923 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.139738 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.140459 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:52.141558 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.142252 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.142829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:52.162096 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf could not be found. Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf HTTP/1.1 Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:52.164091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf could not be found. Oct 04 08:53:52.165469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'b9fd4027-9ae5-4ab2-afc6-c7eb723204cf'} Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf could not be found. Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:52.166222 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf could not be found. Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf could not be found. Oct 04 08:53:52.167691 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:52.169705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance b9fd4027-9ae5-4ab2-afc6-c7eb723204cf could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:52.169705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b851e530-6e8a-45c3-a293-9277bbb65882 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf" status: 404 len: 111 microversion: 2.1 time: 0.047151 Oct 04 08:53:52.169705 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 324/778] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:52 2022] GET /compute/v2.1/servers/b9fd4027-9ae5-4ab2-afc6-c7eb723204cf => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:53:52.180296 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:52.185607 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:52.206975 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.208470 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.208864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:52.211416 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.217629 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.219253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:52.265174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:53:52.497619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-882c1505-259f-4d67-a500-c62afdaa1590 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 204 len: 0 microversion: 2.1 time: 0.322115 Oct 04 08:53:52.504864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 325/779] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:53:52 2022] DELETE /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:53:52.541543 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:52.543389 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:52.571239 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.576094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.579452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:52.583104 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.584399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.590143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:52.887337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:52.914087 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e4dffdcf-c460-409b-8d5b-b6e18892758f tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1288 microversion: 2.1 time: 0.401436 Oct 04 08:53:52.919501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 326/780] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:52 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1288 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:52.930527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:52.931606 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:52.948785 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.949926 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.950735 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:52.951759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:52.952637 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:52.957589 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:53.595127 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:53.637736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-572a6e5d-c6b9-4ef1-a051-7b76ebf1bca8 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1522 microversion: 2.1 time: 0.715887 Oct 04 08:53:53.639358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 327/781] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:52 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1522 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:53.948264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:53.949292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:53.956683 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:53.957752 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:53.958388 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:53.959478 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:53.960137 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:53.960732 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:53.984169 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:54.089466 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:54.108898 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-53da6a31-0647-43fe-bc2f-99ce295a9617 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1294 microversion: 2.1 time: 0.165244 Oct 04 08:53:54.109774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 328/782] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:53 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1294 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:54.224542 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-7bd5cdaf-1624-4d99-8f54-21de2b44df30 req-d885ed73-920e-43c4-8083-a92fb9e7c804 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f597e06f-510c-46fe-92de-ce4a8dc53841", "name": "network-vif-deleted", "tag": "0e360ecf-b764-45c6-8c4f-8ea15e619e9e"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:54.235676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-7bd5cdaf-1624-4d99-8f54-21de2b44df30 req-d885ed73-920e-43c4-8083-a92fb9e7c804 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:54.236426 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-7bd5cdaf-1624-4d99-8f54-21de2b44df30 req-d885ed73-920e-43c4-8083-a92fb9e7c804 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:54.237069 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-7bd5cdaf-1624-4d99-8f54-21de2b44df30 req-d885ed73-920e-43c4-8083-a92fb9e7c804 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:54.252784 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-7bd5cdaf-1624-4d99-8f54-21de2b44df30 req-d885ed73-920e-43c4-8083-a92fb9e7c804 service nova] Creating event network-vif-deleted:0e360ecf-b764-45c6-8c4f-8ea15e619e9e for instance f597e06f-510c-46fe-92de-ce4a8dc53841 on hv-cinder-86020 Oct 04 08:53:54.260232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-7bd5cdaf-1624-4d99-8f54-21de2b44df30 req-d885ed73-920e-43c4-8083-a92fb9e7c804 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039913 Oct 04 08:53:54.261182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 329/783] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:53:54.711202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:54.711202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:54.725979 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:54.727263 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:54.728652 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:54.732718 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:54.733912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:54.735205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:54.886287 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:54.887105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:54.887781 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:54.888781 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lazy-loading 'fault' on Instance uuid f597e06f-510c-46fe-92de-ce4a8dc53841 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:53:55.011984 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '26abb94392364bdc82f6bb508a611d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5934621267af4b838889fcbfb9febaea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:55.026917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9f11516e-317d-4947-b3f6-ada39f72f1c1 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 200 len: 1312 microversion: 2.1 time: 0.336559 Oct 04 08:53:55.030243 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 330/784] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:54 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 1312 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:53:55.161763 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:55.162384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:55.177312 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:55.177808 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:55.178250 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:55.179067 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:55.184522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:55.185205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:55.633774 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:55.642078 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c91a1486-a3ac-43b1-9b2c-7d4217b13344 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1362 microversion: 2.1 time: 0.484654 Oct 04 08:53:55.642736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 331/785] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:55 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1362 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:56.091339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:56.093845 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] [instance: f597e06f-510c-46fe-92de-ce4a8dc53841] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:56.129866 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:56.132906 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:56.133758 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:56.135753 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:56.136570 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:56.137305 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:56.205037 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] HTTP exception thrown: Instance f597e06f-510c-46fe-92de-ce4a8dc53841 could not be found. Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Request method failure captured: Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 HTTP/1.1 Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_BPTCpEq0x-LK1muO3zyHamF0ZYnMzRi2oIVVc-OFyZh1pYpCUuQ6DhUVFefGrdS5RqplwX3pDsBb4d1Y-225IDetn9PJrcL_z3pPMpHCairlrAi5TuXBsy7g18Iw-ElctckOAwPl1bfcgP6056kjXUoXlTLLrv_ywSylIc3pON4ek Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:53:56.209993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5934621267af4b838889fcbfb9febaea"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5934621267af4b838889fcbfb9febaea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5934621267af4b838889fcbfb9febaea", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5934621267af4b838889fcbfb9febaea Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1423515983 Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 26abb94392364bdc82f6bb508a611d44 Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1423515983-project Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance f597e06f-510c-46fe-92de-ce4a8dc53841 could not be found. Oct 04 08:53:56.217176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'f597e06f-510c-46fe-92de-ce4a8dc53841'} Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f597e06f-510c-46fe-92de-ce4a8dc53841 could not be found. Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:53:56.219834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f597e06f-510c-46fe-92de-ce4a8dc53841 could not be found. Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f597e06f-510c-46fe-92de-ce4a8dc53841 could not be found. Oct 04 08:53:56.222950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:53:56.226038 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] Returning 404 to user: Instance f597e06f-510c-46fe-92de-ce4a8dc53841 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:53:56.226038 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ce99b5bf-d403-4852-bd11-395bc597d909 tempest-ServerRescueNegativeTestJSON-1423515983 tempest-ServerRescueNegativeTestJSON-1423515983-project] 10.222.0.16 "GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841" status: 404 len: 111 microversion: 2.1 time: 0.130817 Oct 04 08:53:56.226038 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 332/786] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:56 2022] GET /compute/v2.1/servers/f597e06f-510c-46fe-92de-ce4a8dc53841 => generated 111 bytes in 139 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:53:56.453793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-b1736155-c77f-4cb8-bf9e-91ed67a4b7ce req-59b959b9-e4bf-4bf8-91fa-efd33f41583a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c", "tag": "fc06bc67-05e9-40bf-b9d1-db211978048b"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:53:56.488182 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-b1736155-c77f-4cb8-bf9e-91ed67a4b7ce req-59b959b9-e4bf-4bf8-91fa-efd33f41583a service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:56.490790 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-b1736155-c77f-4cb8-bf9e-91ed67a4b7ce req-59b959b9-e4bf-4bf8-91fa-efd33f41583a service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:56.493244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-b1736155-c77f-4cb8-bf9e-91ed67a4b7ce req-59b959b9-e4bf-4bf8-91fa-efd33f41583a service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:56.523151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-b1736155-c77f-4cb8-bf9e-91ed67a4b7ce req-59b959b9-e4bf-4bf8-91fa-efd33f41583a service nova] Creating event network-changed:fc06bc67-05e9-40bf-b9d1-db211978048b for instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c on hv-cinder-86020 Oct 04 08:53:56.539195 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-b1736155-c77f-4cb8-bf9e-91ed67a4b7ce req-59b959b9-e4bf-4bf8-91fa-efd33f41583a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091362 Oct 04 08:53:56.540589 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 333/787] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:53:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:53:56.670652 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:56.672239 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:56.690243 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:56.694253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:56.695409 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:56.709880 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:56.716055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:56.717005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:57.161008 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:57.171580 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4a016864-b39a-4b90-8ed2-e5a52a820df8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1362 microversion: 2.1 time: 0.506727 Oct 04 08:53:57.173085 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 334/788] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:56 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1362 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:58.073937 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-2f1e0b9c-d502-4433-8414-e5578c6ec7a3 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a076e310>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:58.213685 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-2f1e0b9c-d502-4433-8414-e5578c6ec7a3 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/7a33e468-b4e9-4c09-aeed-94f23b8a8592" status: 200 len: 350 microversion: 2.1 time: 0.144502 Oct 04 08:53:58.215809 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 335/789] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:53:58 2022] GET /compute/v2.1/os-volumes/7a33e468-b4e9-4c09-aeed-94f23b8a8592 => generated 350 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:53:58.224994 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:58.229210 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:58.248032 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:58.248544 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:58.248991 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:58.249725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:58.250189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:58.250660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:58.370123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:58.781431 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:53:58.791910 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9f160dd4-8b93-444a-b065-36e26054a8f8 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1362 microversion: 2.1 time: 0.574247 Oct 04 08:53:58.812196 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 336/790] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:58 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1362 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:53:59.819476 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:53:59.821462 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:53:59.833629 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:59.834809 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:59.841310 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:53:59.842664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:53:59.843467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:53:59.843915 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:00.126039 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=12 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:00.181631 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:54:00.206802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-7a38da97-596d-4ff6-9dd5-550c0875fab6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1362 microversion: 2.1 time: 0.390375 Oct 04 08:54:00.209442 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 337/791] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:53:59 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1362 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:54:00.240261 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-28690ed4-3dd3-426a-a6c0-4f216d39700b tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a07220d0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:00.382831 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-28690ed4-3dd3-426a-a6c0-4f216d39700b tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/3991ead4-d1e2-4618-b820-89a3c88fe5ed" status: 200 len: 351 microversion: 2.1 time: 0.170899 Oct 04 08:54:00.384437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 338/792] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:54:00 2022] GET /compute/v2.1/os-volumes/3991ead4-d1e2-4618-b820-89a3c88fe5ed => generated 351 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:01.242108 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:01.245305 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:01.264089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:01.265495 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:01.267106 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:01.268346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:01.270098 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:01.273812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:01.695829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:54:01.711928 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f130c99e-9ebd-47f9-9815-1657be5491c6 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1362 microversion: 2.1 time: 0.476220 Oct 04 08:54:01.714040 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 339/793] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:54:01 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1362 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:54:01.907578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-52c7fa7a-f747-4543-8215-8822c05b0988 req-854d2795-d899-49f6-88eb-ad11eb2b379e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c", "name": "network-vif-plugged", "status": "completed", "tag": "fc06bc67-05e9-40bf-b9d1-db211978048b"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:54:01.936710 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-52c7fa7a-f747-4543-8215-8822c05b0988 req-854d2795-d899-49f6-88eb-ad11eb2b379e service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:01.938260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-52c7fa7a-f747-4543-8215-8822c05b0988 req-854d2795-d899-49f6-88eb-ad11eb2b379e service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:01.940261 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-52c7fa7a-f747-4543-8215-8822c05b0988 req-854d2795-d899-49f6-88eb-ad11eb2b379e service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:01.970945 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-52c7fa7a-f747-4543-8215-8822c05b0988 req-854d2795-d899-49f6-88eb-ad11eb2b379e service nova] Creating event network-vif-plugged:fc06bc67-05e9-40bf-b9d1-db211978048b for instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c on hv-cinder-86020 Oct 04 08:54:01.981551 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-52c7fa7a-f747-4543-8215-8822c05b0988 req-854d2795-d899-49f6-88eb-ad11eb2b379e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082788 Oct 04 08:54:01.985238 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 340/794] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:54:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:02.077922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e21c3c6e-1f2c-4d71-95b4-d872045b7c25 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a07cf670>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:02.152990 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e21c3c6e-1f2c-4d71-95b4-d872045b7c25 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/8e83cb08-9193-46c4-83f3-c261b7085e73" status: 200 len: 350 microversion: 2.1 time: 0.079672 Oct 04 08:54:02.163901 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 341/795] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:54:02 2022] GET /compute/v2.1/os-volumes/8e83cb08-9193-46c4-83f3-c261b7085e73 => generated 350 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:02.175852 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-137061c1-d052-483f-ad94-e66b395af6c8 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a0722c10>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:02.293778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-137061c1-d052-483f-ad94-e66b395af6c8 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.127546 Oct 04 08:54:02.295023 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 342/796] 10.222.0.16 () {58 vars in 1134 bytes} [Tue Oct 4 08:54:02 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:54:02.311853 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c29b8bed-b082-4e7b-a3c0-143efee00f98 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a076e160>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:02.467869 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c29b8bed-b082-4e7b-a3c0-143efee00f98 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.159555 Oct 04 08:54:02.469348 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 343/797] 10.222.0.16 () {58 vars in 1149 bytes} [Tue Oct 4 08:54:02 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:02.487703 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-966e2ad6-e9e3-4cb8-a3cc-dc9e4c9e73cb tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a076e550>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:02.595896 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-966e2ad6-e9e3-4cb8-a3cc-dc9e4c9e73cb tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.114618 Oct 04 08:54:02.598662 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 344/798] 10.222.0.16 () {58 vars in 1134 bytes} [Tue Oct 4 08:54:02 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:54:02.623514 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c77c17dc-58eb-477d-9d45-90ca24c067ce tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a075e8b0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:02.754848 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c77c17dc-58eb-477d-9d45-90ca24c067ce tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.137141 Oct 04 08:54:02.756415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 345/799] 10.222.0.16 () {58 vars in 1167 bytes} [Tue Oct 4 08:54:02 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:02.770855 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:02.772812 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:02.791140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:02.792316 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:02.793865 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:02.796253 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:02.796918 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:02.797893 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:03.054535 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:54:03.065392 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-44a5c80e-2828-44df-8e2f-341b67ece8d7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1362 microversion: 2.1 time: 0.305682 Oct 04 08:54:03.078793 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 346/800] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:54:02 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1362 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:54:03.090033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-29a2eeb3-7b10-4b9a-ad26-bd83b9e33dc4 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a06e0c10>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:03.233549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-29a2eeb3-7b10-4b9a-ad26-bd83b9e33dc4 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.165787 Oct 04 08:54:03.234665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 347/801] 10.222.0.16 () {58 vars in 1170 bytes} [Tue Oct 4 08:54:03 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:03.254128 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fedbbfd9-f133-47b7-90f9-e7d48e3a015e tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a0722af0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:03.359997 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fedbbfd9-f133-47b7-90f9-e7d48e3a015e tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.111759 Oct 04 08:54:03.361001 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 348/802] 10.222.0.16 () {58 vars in 1155 bytes} [Tue Oct 4 08:54:03 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:54:03.378654 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-063831bf-6826-4366-afe3-95c1103bd5b7 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a07cf310>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:03.558516 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-063831bf-6826-4366-afe3-95c1103bd5b7 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.186442 Oct 04 08:54:03.559308 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 349/803] 10.222.0.16 () {58 vars in 1188 bytes} [Tue Oct 4 08:54:03 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:03.580417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cb6a00b2-68b9-4225-9970-9c457d41f7f1 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] Calling method '.version_select at 0x7f30a06e0d30>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:03.777286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cb6a00b2-68b9-4225-9970-9c457d41f7f1 tempest-VolumesTestJSON-69046806 tempest-VolumesTestJSON-69046806-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.205027 Oct 04 08:54:03.778707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 350/804] 10.222.0.16 () {58 vars in 1155 bytes} [Tue Oct 4 08:54:03 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:54:04.108766 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:04.111103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:04.130371 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:04.131933 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:04.132946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:04.134580 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:04.135856 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:04.136897 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:04.388091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:54:04.403618 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-47bc66d0-792c-4b5f-b144-448e076515f7 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1549 microversion: 2.1 time: 0.310499 Oct 04 08:54:04.404632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 351/805] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:54:04 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1549 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:54:04.795646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-174a3985-ebff-4c4a-92f4-a3ce859968f3 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Action: 'create', calling method: .version_select at 0x7f30a07cf280>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-486224437", "metadata": {"Type": "work"}}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-174a3985-ebff-4c4a-92f4-a3ce859968f3 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Request method failure captured: Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 121 Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_Ssg5OaZ5AFX_RLrIQWKNdhNmM6k522rQHu_5w0eUMF9e_syp_C3MzMyCbwqOgSKBeZJyFR-w0B5-ZxoUStvr5dwyb2QHosOTdXxzddmudspqmdsUDJjhAPcqbsXZB96SaP6eHOlsaCVx8GUCKqUT5YFVVLbhTYLOIwkO0TMBqKXv4 Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:54:04.811594 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_00fd9ac2b3434dd789ae4d53d93fa80c", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 6b28bc14e2324d10a0ca8ab720bbf1fc Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-486224437", "metadata": {"Type": "work"}}} Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:54:04.813129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-486224437', 'metadata': {'Type': 'work'}}}} Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise error Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi '#$%' Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:54:04.813989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 04 08:54:04.816002 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 04 08:54:04.817265 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 04 08:54:04.817265 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.817265 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-174a3985-ebff-4c4a-92f4-a3ce859968f3 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:54:04.817265 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-174a3985-ebff-4c4a-92f4-a3ce859968f3 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.408277 Oct 04 08:54:04.817265 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 352/806] 10.222.0.16 () {60 vars in 1156 bytes} [Tue Oct 4 08:54:04 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 410 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:54:04.821161 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:04.821661 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:04.830861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:04.832333 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:04.832861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:04.833835 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:04.834523 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:04.834992 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:04.928441 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0edf1677-4079-43e6-8f5c-6343478477ca tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.111340 Oct 04 08:54:04.929052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 353/807] 10.222.0.16 () {58 vars in 1275 bytes} [Tue Oct 4 08:54:04 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-interface => generated 285 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:04.934355 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9aca9fd4-e11f-4e19-ab2b-f8a619a5d2f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Action: 'create', calling method: .version_select at 0x7f30a06e0550>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1698536839", "metadata": {"Type": "work"}}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9aca9fd4-e11f-4e19-ab2b-f8a619a5d2f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Request method failure captured: Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 120 Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_Ssg5OaZ5AFX_RLrIQWKNdhNmM6k522rQHu_5w0eUMF9e_syp_C3MzMyCbwqOgSKBeZJyFR-w0B5-ZxoUStvr5dwyb2QHosOTdXxzddmudspqmdsUDJjhAPcqbsXZB96SaP6eHOlsaCVx8GUCKqUT5YFVVLbhTYLOIwkO0TMBqKXv4 Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:54:04.940782 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_00fd9ac2b3434dd789ae4d53d93fa80c", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 6b28bc14e2324d10a0ca8ab720bbf1fc Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1698536839", "metadata": {"Type": "work"}}} Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:54:04.942988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1698536839', 'metadata': {'Type': 'work'}}}} Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise error Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi '0' Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:54:04.944027 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 04 08:54:04.947372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 04 08:54:04.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 04 08:54:04.950112 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.951836 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9aca9fd4-e11f-4e19-ab2b-f8a619a5d2f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:54:04.952979 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9aca9fd4-e11f-4e19-ab2b-f8a619a5d2f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.022686 Oct 04 08:54:04.953813 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 354/808] 10.222.0.16 () {60 vars in 1156 bytes} [Tue Oct 4 08:54:04 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 24 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:54:04.966380 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-33dbd212-31c7-4466-9671-fa270ca34766 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Action: 'create', calling method: .version_select at 0x7f30a076e820>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2006210976", "metadata": {"Type": "work"}}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-33dbd212-31c7-4466-9671-fa270ca34766 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Request method failure captured: Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 119 Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_Ssg5OaZ5AFX_RLrIQWKNdhNmM6k522rQHu_5w0eUMF9e_syp_C3MzMyCbwqOgSKBeZJyFR-w0B5-ZxoUStvr5dwyb2QHosOTdXxzddmudspqmdsUDJjhAPcqbsXZB96SaP6eHOlsaCVx8GUCKqUT5YFVVLbhTYLOIwkO0TMBqKXv4 Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:54:04.973448 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_00fd9ac2b3434dd789ae4d53d93fa80c", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 6b28bc14e2324d10a0ca8ab720bbf1fc Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2006210976", "metadata": {"Type": "work"}}} Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:54:04.975555 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-2006210976', 'metadata': {'Type': 'work'}}}} Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise error Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi '' Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:54:04.976437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 04 08:54:04.977996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 04 08:54:04.981041 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 04 08:54:04.981041 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:04.982419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-33dbd212-31c7-4466-9671-fa270ca34766 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:54:04.984256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-33dbd212-31c7-4466-9671-fa270ca34766 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.022640 Oct 04 08:54:04.985193 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 355/809] 10.222.0.16 () {60 vars in 1156 bytes} [Tue Oct 4 08:54:04 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 24 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 04 08:54:04.999066 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-363a3fa7-0738-4510-9d17-5dd465f00261 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Calling method '.version_select at 0x7f30a075e8b0>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:05.041274 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-363a3fa7-0738-4510-9d17-5dd465f00261 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] HTTP exception thrown: Volume tempest-invalid-269110399 could not be found. Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-363a3fa7-0738-4510-9d17-5dd465f00261 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Request method failure captured: Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-269110399 HTTP/1.1 Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_Ssg5OaZ5AFX_RLrIQWKNdhNmM6k522rQHu_5w0eUMF9e_syp_C3MzMyCbwqOgSKBeZJyFR-w0B5-ZxoUStvr5dwyb2QHosOTdXxzddmudspqmdsUDJjhAPcqbsXZB96SaP6eHOlsaCVx8GUCKqUT5YFVVLbhTYLOIwkO0TMBqKXv4 Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:54:05.047153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_00fd9ac2b3434dd789ae4d53d93fa80c", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 6b28bc14e2324d10a0ca8ab720bbf1fc Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Volume tempest-invalid-269110399 could not be found. Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'tempest-invalid-269110399'} Oct 04 08:54:05.051130 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-269110399 could not be found. (HTTP 404) (Request-ID: req-e97539aa-cf79-4da7-891e-0f022f1eb35f) Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.053292 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 04 08:54:05.056172 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-269110399 could not be found. Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:54:05.060387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-269110399 could not be found. Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-269110399 could not be found. Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-363a3fa7-0738-4510-9d17-5dd465f00261 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Returning 404 to user: Volume tempest-invalid-269110399 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:54:05.061586 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-363a3fa7-0738-4510-9d17-5dd465f00261 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/tempest-invalid-269110399" status: 404 len: 98 microversion: 2.1 time: 0.065831 Oct 04 08:54:05.063431 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 356/810] 10.222.0.16 () {58 vars in 1215 bytes} [Tue Oct 4 08:54:04 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-269110399 => generated 98 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Oct 04 08:54:05.081863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4069a799-8a86-4094-aee5-16b54114ce9a tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.009601 Oct 04 08:54:05.082632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 357/811] 10.222.0.16 () {58 vars in 1140 bytes} [Tue Oct 4 08:54:05 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 13 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 04 08:54:05.106289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c5329f73-cf0e-4754-9e91-98eaaaa5d211 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.018274 Oct 04 08:54:05.113804 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 358/812] 10.222.0.16 () {58 vars in 1137 bytes} [Tue Oct 4 08:54:05 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 27 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 04 08:54:05.126989 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-55a7322d-14d5-434a-89de-cfa41563286a tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Calling method '.version_select at 0x7f30a06f1b80>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:05.182340 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-55a7322d-14d5-434a-89de-cfa41563286a tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] HTTP exception thrown: Volume e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 could not be found. Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-55a7322d-14d5-434a-89de-cfa41563286a tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Request method failure captured: Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/os-volumes/e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 HTTP/1.1 Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_Ssg5OaZ5AFX_RLrIQWKNdhNmM6k522rQHu_5w0eUMF9e_syp_C3MzMyCbwqOgSKBeZJyFR-w0B5-ZxoUStvr5dwyb2QHosOTdXxzddmudspqmdsUDJjhAPcqbsXZB96SaP6eHOlsaCVx8GUCKqUT5YFVVLbhTYLOIwkO0TMBqKXv4 Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:54:05.185110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_00fd9ac2b3434dd789ae4d53d93fa80c", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 6b28bc14e2324d10a0ca8ab720bbf1fc Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Volume e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 could not be found. Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'e6c374a9-e11d-4a0f-87d4-edc356bb5bf0'} Oct 04 08:54:05.186591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 could not be found. (HTTP 404) (Request-ID: req-3e1fb61b-e341-4b6b-9caa-90c87803d46c) Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.187961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 04 08:54:05.189277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 could not be found. Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 04 08:54:05.191376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 could not be found. Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 could not be found. Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-55a7322d-14d5-434a-89de-cfa41563286a tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Returning 404 to user: Volume e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-55a7322d-14d5-434a-89de-cfa41563286a tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/e6c374a9-e11d-4a0f-87d4-edc356bb5bf0" status: 404 len: 109 microversion: 2.1 time: 0.067509 Oct 04 08:54:05.192724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 359/813] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:54:05 2022] DELETE /compute/v2.1/os-volumes/e6c374a9-e11d-4a0f-87d4-edc356bb5bf0 => generated 109 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:54:05.203765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4f7fbfad-5ea0-4d99-a3ba-e76a4a8243f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Calling method '.version_select at 0x7f30a076eb80>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:05.265725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-4f7fbfad-5ea0-4d99-a3ba-e76a4a8243f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] HTTP exception thrown: Volume cf98c121-62f2-4ade-8b90-4c50259b0701 could not be found. Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4f7fbfad-5ea0-4d99-a3ba-e76a4a8243f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Request method failure captured: Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/os-volumes/cf98c121-62f2-4ade-8b90-4c50259b0701 HTTP/1.1 Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_Ssg5OaZ5AFX_RLrIQWKNdhNmM6k522rQHu_5w0eUMF9e_syp_C3MzMyCbwqOgSKBeZJyFR-w0B5-ZxoUStvr5dwyb2QHosOTdXxzddmudspqmdsUDJjhAPcqbsXZB96SaP6eHOlsaCVx8GUCKqUT5YFVVLbhTYLOIwkO0TMBqKXv4 Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:54:05.271584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/00fd9ac2b3434dd789ae4d53d93fa80c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_00fd9ac2b3434dd789ae4d53d93fa80c", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 00fd9ac2b3434dd789ae4d53d93fa80c Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesNegativeTest-1199298497 Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 6b28bc14e2324d10a0ca8ab720bbf1fc Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesNegativeTest-1199298497-project Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Volume cf98c121-62f2-4ade-8b90-4c50259b0701 could not be found. Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'cf98c121-62f2-4ade-8b90-4c50259b0701'} Oct 04 08:54:05.274677 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume cf98c121-62f2-4ade-8b90-4c50259b0701 could not be found. (HTTP 404) (Request-ID: req-e050cdfc-b1ba-45c2-9a39-0e279d83c020) Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.276824 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 04 08:54:05.280657 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume cf98c121-62f2-4ade-8b90-4c50259b0701 could not be found. Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 04 08:54:05.283415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume cf98c121-62f2-4ade-8b90-4c50259b0701 could not be found. Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume cf98c121-62f2-4ade-8b90-4c50259b0701 could not be found. Oct 04 08:54:05.286061 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:54:05.294659 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-4f7fbfad-5ea0-4d99-a3ba-e76a4a8243f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] Returning 404 to user: Volume cf98c121-62f2-4ade-8b90-4c50259b0701 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:54:05.298041 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-4f7fbfad-5ea0-4d99-a3ba-e76a4a8243f2 tempest-VolumesNegativeTest-1199298497 tempest-VolumesNegativeTest-1199298497-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/cf98c121-62f2-4ade-8b90-4c50259b0701" status: 404 len: 109 microversion: 2.1 time: 0.101532 Oct 04 08:54:05.300435 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 360/814] 10.222.0.16 () {58 vars in 1245 bytes} [Tue Oct 4 08:54:05 2022] GET /compute/v2.1/os-volumes/cf98c121-62f2-4ade-8b90-4c50259b0701 => generated 109 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:54:07.208666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-86e07977-4b77-450b-b0f9-20b25ab79ef7 req-2d27a256-2882-4b86-a58d-a7304098e135 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c", "tag": "fc06bc67-05e9-40bf-b9d1-db211978048b"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:54:07.223501 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-86e07977-4b77-450b-b0f9-20b25ab79ef7 req-2d27a256-2882-4b86-a58d-a7304098e135 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:07.224347 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-86e07977-4b77-450b-b0f9-20b25ab79ef7 req-2d27a256-2882-4b86-a58d-a7304098e135 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:07.225076 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-86e07977-4b77-450b-b0f9-20b25ab79ef7 req-2d27a256-2882-4b86-a58d-a7304098e135 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:07.245190 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-86e07977-4b77-450b-b0f9-20b25ab79ef7 req-2d27a256-2882-4b86-a58d-a7304098e135 service nova] Creating event network-changed:fc06bc67-05e9-40bf-b9d1-db211978048b for instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c on hv-cinder-86020 Oct 04 08:54:07.251898 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-86e07977-4b77-450b-b0f9-20b25ab79ef7 req-2d27a256-2882-4b86-a58d-a7304098e135 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048240 Oct 04 08:54:07.258259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 361/815] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:54:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:37.469247 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=4,nova_api:UPDATE=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:37.469944 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=67 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:37.476442 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:37.477185 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:37.485927 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:37.486571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:37.487016 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:37.487753 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:37.488220 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:37.488646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:37.518665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ccc61c05-84ff-4cc3-8156-33ace6a4c5de tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/ips" status: 200 len: 146 microversion: 2.1 time: 0.050600 Oct 04 08:54:37.519547 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 362/816] 10.222.0.16 () {58 vars in 1248 bytes} [Tue Oct 4 08:54:37 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/ips => generated 146 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:39.998107 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cab09ab6-0594-415a-80c4-d6b3d89043f7"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:54:40.000612 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:40.008063 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:40.008561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:40.009005 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:40.009725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:40.010187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:40.010660 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:40.352529 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6c461799-38ec-4631-b1d0-67b553a00056 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.358437 Oct 04 08:54:40.353276 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 363/817] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:54:39 2022] POST /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments => generated 194 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:47.987777 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:47.990488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:47.997329 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:47.997823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:47.998265 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:47.999051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:47.999502 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:48.000010 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:48.035779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-e3d503c3-a91b-4765-bd3e-39a0a1e9562e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054111 Oct 04 08:54:48.036372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 364/818] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:54:47 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:48.050246 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:48.050776 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:48.057159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:48.057648 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:48.058080 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:48.059597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:48.060208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:48.060629 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:48.096966 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9319e83b-c34f-465c-9d4c-abd6fa02ab1e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/cab09ab6-0594-415a-80c4-d6b3d89043f7" status: 200 len: 194 microversion: 2.1 time: 0.052341 Oct 04 08:54:48.097596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 365/819] 10.222.0.16 () {58 vars in 1413 bytes} [Tue Oct 4 08:54:48 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/cab09ab6-0594-415a-80c4-d6b3d89043f7 => generated 194 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:54:50.582860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "02372640-eb63-4f12-b990-7352390ad7d2"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:54:50.585235 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:50.591929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:50.592430 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:50.593115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:50.593846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:50.594334 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:50.595443 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:50.894105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-13db1a95-039b-4f15-85bb-e10752b71859 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "POST /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.316396 Oct 04 08:54:50.894780 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 366/820] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:54:50 2022] POST /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments => generated 194 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:58.500122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:58.502380 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:58.509215 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:58.509695 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:58.510129 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:58.510843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:58.511317 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:58.511743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:58.541858 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f519e747-7b21-42d7-9cf0-910c374b53ec tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.046553 Oct 04 08:54:58.542415 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 367/821] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:54:58 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments => generated 371 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:58.686951 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:58.687503 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:58.693743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:58.694208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:58.694664 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:58.695354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:58.695798 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:58.696212 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:58.726579 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3feff4b9-4627-4969-a8b3-6c4f49ec1d04 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/cab09ab6-0594-415a-80c4-d6b3d89043f7" status: 200 len: 194 microversion: 2.1 time: 0.044485 Oct 04 08:54:58.727199 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 368/822] 10.222.0.16 () {58 vars in 1413 bytes} [Tue Oct 4 08:54:58 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/cab09ab6-0594-415a-80c4-d6b3d89043f7 => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:54:58.739169 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:54:58.739781 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:54:58.745826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:58.746303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:58.746776 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:58.747461 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:54:58.747908 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:54:58.749016 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:54:58.839828 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lazy-loading 'services' on Instance uuid ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:54:58.921742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0b6c5e71-9156-48eb-bea7-76efc1b2c24b tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/cab09ab6-0594-415a-80c4-d6b3d89043f7" status: 202 len: 0 microversion: 2.1 time: 0.187541 Oct 04 08:54:58.922522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 369/823] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:54:58 2022] DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/cab09ab6-0594-415a-80c4-d6b3d89043f7 => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:55:00.080644 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:00.081338 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:00.088241 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:00.088916 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:00.089357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:00.090186 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:00.090807 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:00.091233 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:00.130887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-bcc6e7a7-835a-439c-91d6-0a2285cad4cf tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/02372640-eb63-4f12-b990-7352390ad7d2" status: 200 len: 194 microversion: 2.1 time: 0.054878 Oct 04 08:55:00.131739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 370/824] 10.222.0.16 () {58 vars in 1413 bytes} [Tue Oct 4 08:55:00 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/02372640-eb63-4f12-b990-7352390ad7d2 => generated 194 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:55:00.147055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:00.147055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:00.152204 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:00.152976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:00.153718 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:00.154446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:00.154937 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:00.155507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:00.254219 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lazy-loading 'services' on Instance uuid ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:55:00.336433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b1d189a7-4da8-4c06-9489-802985ade6f4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/02372640-eb63-4f12-b990-7352390ad7d2" status: 202 len: 0 microversion: 2.1 time: 0.197528 Oct 04 08:55:00.337571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 371/825] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:55:00 2022] DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments/02372640-eb63-4f12-b990-7352390ad7d2 => generated 0 bytes in 199 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:55:01.552998 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:01.555324 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:01.561507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:01.562069 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:01.562874 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:01.564117 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:01.564942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:01.565899 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:01.600229 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-cbc2d691-decf-4662-bfd7-3d528997c940 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053853 Oct 04 08:55:01.600802 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 372/826] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:55:01 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 04 08:55:01.729620 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:01.732295 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:01.739602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:01.740210 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:01.740694 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:01.741527 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:01.742012 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:01.742452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:01.775712 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-988eafda-ef28-431a-802d-d80c9a4b7f0a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055966 Oct 04 08:55:01.776307 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 373/827] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:55:01 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c/os-volume_attachments => generated 25 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:55:01.851326 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:01.851820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:01.858695 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:01.859268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:01.859700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:01.860397 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:01.860843 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:01.861259 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:01.895068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:55:01.898407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=4 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:01.992100 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-fccdc578-8361-4c10-876a-77f66908a53e tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 204 len: 0 microversion: 2.1 time: 0.145281 Oct 04 08:55:01.992889 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 374/828] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:55:01 2022] DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 04 08:55:02.008825 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:02.010151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:02.021755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:02.022263 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:02.022736 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:02.023482 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:02.023934 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:02.024360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:02.178404 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:02.186952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-03b0fc6a-ce5a-46cd-9104-434a78f2092d tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1673 microversion: 2.1 time: 0.188577 Oct 04 08:55:02.187539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 375/829] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:01 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1673 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:55:03.212936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:03.213593 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:03.219837 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:03.220346 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:03.220761 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:03.221955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:03.221955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:03.222961 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:03.369986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:03.378795 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c241f2bd-3c19-4ba3-8ac1-6f61c5fa9121 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1673 microversion: 2.1 time: 0.169324 Oct 04 08:55:03.379663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 376/830] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:03 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1673 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:55:04.405483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:04.406947 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:04.413619 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:04.414056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:04.414432 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:04.415589 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:04.416349 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:04.421875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:04.553009 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:04.568859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-275ec0ea-cc90-4f29-a0f6-62000fa7db89 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1605 microversion: 2.1 time: 0.168507 Oct 04 08:55:04.569449 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 377/831] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:04 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1605 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:55:04.773457 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-d2800bf0-6e35-4462-9aed-23178949d599 req-ff5b7fba-4eae-43a7-927c-10588f7eb807 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c", "name": "network-vif-deleted", "tag": "fc06bc67-05e9-40bf-b9d1-db211978048b"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:55:04.788725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-d2800bf0-6e35-4462-9aed-23178949d599 req-ff5b7fba-4eae-43a7-927c-10588f7eb807 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:04.789584 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-d2800bf0-6e35-4462-9aed-23178949d599 req-ff5b7fba-4eae-43a7-927c-10588f7eb807 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:04.790102 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-d2800bf0-6e35-4462-9aed-23178949d599 req-ff5b7fba-4eae-43a7-927c-10588f7eb807 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:04.803713 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-d2800bf0-6e35-4462-9aed-23178949d599 req-ff5b7fba-4eae-43a7-927c-10588f7eb807 service nova] Creating event network-vif-deleted:fc06bc67-05e9-40bf-b9d1-db211978048b for instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c on hv-cinder-86020 Oct 04 08:55:04.810029 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-d2800bf0-6e35-4462-9aed-23178949d599 req-ff5b7fba-4eae-43a7-927c-10588f7eb807 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046307 Oct 04 08:55:04.811201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 378/832] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:55:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:55:05.595203 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:05.595991 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:05.603455 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:05.604406 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:05.604957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:05.605965 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:05.606538 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:05.607105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:05.693126 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:05.703358 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-96d64a09-311d-4701-81b5-babf65be1f20 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1321 microversion: 2.1 time: 0.113353 Oct 04 08:55:05.703958 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 379/833] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:05 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1321 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:55:06.731704 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:06.732502 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:06.740616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:06.741176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:06.741675 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:06.742519 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:06.743135 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:06.743692 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:06.813707 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78634b34107d49cdbf0126014bf8f6f5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d21789971964770bdf417547cd19840', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:06.822225 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f83995ca-1a24-49c4-a5e1-b95fd32e81b1 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 200 len: 1321 microversion: 2.1 time: 0.094383 Oct 04 08:55:06.823269 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 380/834] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:06 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 1321 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:55:07.850562 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:07.851254 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:07.853631 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:07.859056 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:07.859542 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:07.859976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:07.860676 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:07.861115 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:07.861533 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:07.879491 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] HTTP exception thrown: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Request method failure captured: Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c HTTP/1.1 Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_C2_30scAmridThWKmbW4chXmDOUW8CmDpNKeNqJ2RQzuBAKGAhffCKe5-lQA3FxfABPIN2eVqtWUyGUBPonQ0fz8Zk_vT-9Jddfc_ED9G8U327X1gi5nv7wGesdYWJ3Z1saIJjI7TcpD9slv5w1XHI6vZ0o5eIoa4bIW0dwpla5OQ Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:55:07.881510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5d21789971964770bdf417547cd19840"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5d21789971964770bdf417547cd19840", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 78634b34107d49cdbf0126014bf8f6f5 Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:07.882977 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c'} Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:55:07.883829 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:07.885319 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:07.886486 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Returning 404 to user: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:55:07.886486 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-f2ba9957-5803-4035-b56c-60a276a2766a tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 404 len: 111 microversion: 2.1 time: 0.038041 Oct 04 08:55:07.886486 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 381/835] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:07 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:55:07.893889 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0cacb498-5a36-4bfa-9093-1aec9a9ea211 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '.version_select at 0x7f30a06ab550>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:07.914536 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0cacb498-5a36-4bfa-9093-1aec9a9ea211 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-35649159" status: 202 len: 0 microversion: 2.1 time: 0.024311 Oct 04 08:55:07.915199 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 382/836] 10.222.0.16 () {58 vars in 1215 bytes} [Tue Oct 4 08:55:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-35649159 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 04 08:55:11.204714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:11.205400 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:11.212699 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:11.213211 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:11.213724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:11.214420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:11.215059 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:11.215646 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:11.230820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] HTTP exception thrown: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Request method failure captured: Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c HTTP/1.1 Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_C2_30scAmridThWKmbW4chXmDOUW8CmDpNKeNqJ2RQzuBAKGAhffCKe5-lQA3FxfABPIN2eVqtWUyGUBPonQ0fz8Zk_vT-9Jddfc_ED9G8U327X1gi5nv7wGesdYWJ3Z1saIJjI7TcpD9slv5w1XHI6vZ0o5eIoa4bIW0dwpla5OQ Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:55:11.232739 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5d21789971964770bdf417547cd19840"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5d21789971964770bdf417547cd19840", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 78634b34107d49cdbf0126014bf8f6f5 Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.234354 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c'} Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:55:11.235291 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.236769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.239806 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Returning 404 to user: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:55:11.240674 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-3b5571ea-9455-4d19-bbd1-037edeb5e4a4 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 404 len: 111 microversion: 2.1 time: 0.040826 Oct 04 08:55:11.241251 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 383/837] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:55:11 2022] DELETE /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:55:11.249546 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:11.250163 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:11.257309 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:11.257796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:11.258234 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:11.258979 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:11.259439 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:11.259864 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:11.276770 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] HTTP exception thrown: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Request method failure captured: Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c HTTP/1.1 Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_C2_30scAmridThWKmbW4chXmDOUW8CmDpNKeNqJ2RQzuBAKGAhffCKe5-lQA3FxfABPIN2eVqtWUyGUBPonQ0fz8Zk_vT-9Jddfc_ED9G8U327X1gi5nv7wGesdYWJ3Z1saIJjI7TcpD9slv5w1XHI6vZ0o5eIoa4bIW0dwpla5OQ Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:55:11.278303 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5d21789971964770bdf417547cd19840"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5d21789971964770bdf417547cd19840", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 78634b34107d49cdbf0126014bf8f6f5 Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.279922 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': 'ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c'} Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:55:11.280801 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. Oct 04 08:55:11.282289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:11.284110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Returning 404 to user: Instance ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:55:11.284110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-48a9a46d-99f3-46bf-a4c2-7d7ff82c9796 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c" status: 404 len: 111 microversion: 2.1 time: 0.034420 Oct 04 08:55:11.284110 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 384/838] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:11 2022] GET /compute/v2.1/servers/ce5804f0-e7e9-4932-80a9-4e1c6e7c8f5c => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:55:12.449609 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:12.451008 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:12.459437 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:12.460051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:12.460453 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:12.461510 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:12.461985 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:12.462417 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:12.478242 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] HTTP exception thrown: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Request method failure captured: Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a HTTP/1.1 Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_C2_30scAmridThWKmbW4chXmDOUW8CmDpNKeNqJ2RQzuBAKGAhffCKe5-lQA3FxfABPIN2eVqtWUyGUBPonQ0fz8Zk_vT-9Jddfc_ED9G8U327X1gi5nv7wGesdYWJ3Z1saIJjI7TcpD9slv5w1XHI6vZ0o5eIoa4bIW0dwpla5OQ Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:55:12.480335 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5d21789971964770bdf417547cd19840"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5d21789971964770bdf417547cd19840", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 78634b34107d49cdbf0126014bf8f6f5 Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.481935 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '9606a52d-3c5d-4287-afb7-c84f5e245c9a'} Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:55:12.482826 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.484064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.485796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Returning 404 to user: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:55:12.485796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-6a3d833d-3c2c-49d4-964c-32be2596ab8c tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 404 len: 111 microversion: 2.1 time: 0.036063 Oct 04 08:55:12.485796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 385/839] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:55:12 2022] DELETE /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 04 08:55:12.491846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:12.492616 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] [instance: 9606a52d-3c5d-4287-afb7-c84f5e245c9a] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:12.499020 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:12.499517 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:12.499964 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:12.500768 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:12.501227 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:12.501652 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:12.521033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] HTTP exception thrown: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Request method failure captured: Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a HTTP/1.1 Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_C2_30scAmridThWKmbW4chXmDOUW8CmDpNKeNqJ2RQzuBAKGAhffCKe5-lQA3FxfABPIN2eVqtWUyGUBPonQ0fz8Zk_vT-9Jddfc_ED9G8U327X1gi5nv7wGesdYWJ3Z1saIJjI7TcpD9slv5w1XHI6vZ0o5eIoa4bIW0dwpla5OQ Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:55:12.522602 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5d21789971964770bdf417547cd19840"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5d21789971964770bdf417547cd19840"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5d21789971964770bdf417547cd19840", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 5d21789971964770bdf417547cd19840 Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-AttachVolumeTestJSON-807237847 Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 78634b34107d49cdbf0126014bf8f6f5 Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-AttachVolumeTestJSON-807237847-project Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.524232 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '9606a52d-3c5d-4287-afb7-c84f5e245c9a'} Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:55:12.527539 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. Oct 04 08:55:12.528986 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:55:12.530823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] Returning 404 to user: Instance 9606a52d-3c5d-4287-afb7-c84f5e245c9a could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:55:12.530823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0045a88b-fb10-4b25-8d49-383da259b3f0 tempest-AttachVolumeTestJSON-807237847 tempest-AttachVolumeTestJSON-807237847-project] 10.222.0.16 "GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a" status: 404 len: 111 microversion: 2.1 time: 0.037437 Oct 04 08:55:12.530823 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 386/840] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:12 2022] GET /compute/v2.1/servers/9606a52d-3c5d-4287-afb7-c84f5e245c9a => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:55:40.717197 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.security_group_api [None req-d1a064e8-21fe-44d0-af70-03444e3a0bbd tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Neutron security group tempest-secgroup-smoke-1159980666 not found {{(pid=127142) validate_name /opt/stack/nova/nova/network/security_group_api.py:74}} Oct 04 08:55:40.718392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-d1a064e8-21fe-44d0-af70-03444e3a0bbd tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] HTTP exception thrown: Unable to find security_group with name or id 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d1a064e8-21fe-44d0-af70-03444e3a0bbd tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Request method failure captured: Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: POST /compute/v2.1/servers HTTP/1.1 Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 478 Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_CnSvFrbhlBT_K5TVdjqDTV-KdDvXmqBI8wJ7xyzATzsZ4QqjebkIVG3_kLzU0aaC5tNmuhOQa0f69GHSqFQTsVRqbDA2xykwT-9OZIVyRoHzTckm2cgtb02CmD50md8QV8vX312jxdj7Uk7IxG_lkGl1-MMYrIhQmn0CF5kZccOyU Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:55:40.751337 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2f94e3ae140b43739febe1111e18604b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2f94e3ae140b43739febe1111e18604b", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-TestVolumeBootPattern-371052090 Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 2f94e3ae140b43739febe1111e18604b Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-TestVolumeBootPattern-371052090 Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: ab4b3c2b3f4b4978b0eb472f9c2736b1 Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-TestVolumeBootPattern-371052090-project Oct 04 08:55:40.754096 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {"server": {"name": "tempest-TestVolumeBootPattern-server-1879105048", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2095819453", "security_groups": [{"name": "tempest-secgroup-smoke-1159980666"}], "block_device_mapping_v2": [{"uuid": "1478f59f-9b4d-4eb0-827a-f36c53d04641", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e71b68ee-781f-4e9d-8cf4-0cfc8be1f31a"}]}} Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Unable to find security_group with name or id 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'body': {'server': {'name': 'tempest-TestVolumeBootPattern-server-1879105048', 'imageRef': '', 'flavorRef': '452', 'key_name': 'tempest-TestVolumeBootPattern-2095819453', 'security_groups': [{'name': 'tempest-secgroup-smoke-1159980666'}], 'block_device_mapping_v2': [{'source_type': 'volume', 'destination_type': 'volume', 'boot_index': 0, 'delete_on_termination': False, 'volume_id': '1478f59f-9b4d-4eb0-827a-f36c53d04641'}], 'networks': [{'uuid': 'e71b68ee-781f-4e9d-8cf4-0cfc8be1f31a'}]}}} Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 476, in find_resource Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return self.find_resource_by_id(resource, name_or_id, Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 04 08:55:40.755332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 440, in find_resource_by_id Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with id 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 480, in find_resource Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return self._find_resource_by_name( Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 469, in _find_resource_by_name Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 68, in validate_name Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return neutronv20.find_resourceid_by_name_or_id( Oct 04 08:55:40.756843 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 56, in find_resourceid_by_name_or_id Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return find_resource_by_name_or_id(client, resource, name_or_id, Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 49, in find_resource_by_name_or_id Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return client.find_resource(resource, name_or_id, project_id, Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 488, in find_resource Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound( Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name or id 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return self._create_instance( Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi ) = self._validate_and_build_base_options( Oct 04 08:55:40.759393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1074, in _validate_and_build_base_options Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi security_groups = self._check_requested_secgroups( Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 521, in _check_requested_secgroups Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi secgroup_uuid = security_group_api.validate_name(context, secgroup) Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 75, in validate_name Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Unable to find security_group with name or id 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:55:40.762876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Unable to find security_group with name or id 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Unable to find security_group with name or id 'tempest-secgroup-smoke-1159980666' Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:55:40.768581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d1a064e8-21fe-44d0-af70-03444e3a0bbd tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Returning 400 to user: Unable to find security_group with name or id 'tempest-secgroup-smoke-1159980666' {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:55:40.781537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d1a064e8-21fe-44d0-af70-03444e3a0bbd tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 125 microversion: 2.1 time: 900.652979 Oct 04 08:55:40.781537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:55:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:55:40.781537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:55:40 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/servers (10.222.0.16) Oct 04 08:55:40.781537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: CRITICAL nova [None req-d1a064e8-21fe-44d0-af70-03444e3a0bbd tempest-TestVolumeBootPattern-371052090 tempest-TestVolumeBootPattern-371052090-project] Unhandled error: OSError: write error Oct 04 08:55:40.781537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova OSError: write error Oct 04 08:55:40.781537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova Oct 04 08:55:40.781537 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 455/841] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:40:40 2022] POST /compute/v2.1/servers => generated 0 bytes in 900657 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Oct 04 08:55:56.604386 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-714b8143-e5ec-4b40-a83b-c9d7df44541b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Action: 'create', calling method: .version_select at 0x7f30a069a9d0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-949961484", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG/CjOrozKl+DiaTgEsCIBbUP17QvgIS8JhlI24HbxZ55S35hdcyr7YgV75raPv7C6gw9Mhw+AfCoDelGyRZhu3NY3H5DEnBjGfdguNErqbKcHUk3ZXukTCwvELZTz+XqA=="}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:55:56.610176 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=12,nova_api:DELETE=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:56.611050 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=50,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:56.612236 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Oct 04 08:55:56.631135 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-714b8143-e5ec-4b40-a83b-c9d7df44541b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Getting quotas for project 8a08fa28d7374a029062132a73bb8ab6. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:55:56.639017 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-714b8143-e5ec-4b40-a83b-c9d7df44541b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Getting quotas for user b62e31d5979c426fb5b6416f087692ff and project 8a08fa28d7374a029062132a73bb8ab6. Resources: {'key_pairs'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:55:56.664446 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-714b8143-e5ec-4b40-a83b-c9d7df44541b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.064146 Oct 04 08:55:56.665048 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 387/842] 10.222.0.16 () {60 vars in 1159 bytes} [Tue Oct 4 08:55:56 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:55:56.715719 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:55:56.716558 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:55:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.16) !!! Oct 04 08:55:56.721311 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2108338758", "imageRef": "78026e8b-9cb9-496b-941a-0b1310c0b302", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-949961484", "networks": [{"uuid": "543bb562-32db-4705-a52a-29b509ec3fe1"}]}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:55:56.778600 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.network.neutron [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] validate_networks() for [('543bb562-32db-4705-a52a-29b509ec3fe1', None, None, None, None, None)] {{(pid=127142) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:55:57.346862 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Flavor limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:55:57.347297 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Image limits 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:55:57.347770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Flavor pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:55:57.348185 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Image pref 0:0:0 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:55:57.348595 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.virt.hardware [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127142) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:55:57.357623 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:57.358090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:57.358523 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:57.359287 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:57.359743 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:57.360170 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:57.378996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Getting quotas for project 8a08fa28d7374a029062132a73bb8ab6. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:55:57.383041 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.quota [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Getting quotas for user b62e31d5979c426fb5b6416f087692ff and project 8a08fa28d7374a029062132a73bb8ab6. Resources: {'cores', 'ram', 'instances'} {{(pid=127142) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:55:57.395084 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Going to run 1 instances... {{(pid=127142) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:55:57.397285 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='78026e8b-9cb9-496b-941a-0b1310c0b302',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127142) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:55:57.448133 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-07d6b0e8-f395-44ae-9886-ac72caa6a9a3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.734172 Oct 04 08:55:57.448781 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 456/843] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:55:56 2022] POST /compute/v2.1/servers => generated 380 bytes in 736 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 04 08:55:57.460340 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-62b8fb92-84cb-482f-94ba-ec05b3bb5258 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:57.461068 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-62b8fb92-84cb-482f-94ba-ec05b3bb5258 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:57.539648 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-62b8fb92-84cb-482f-94ba-ec05b3bb5258 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:57.547289 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-62b8fb92-84cb-482f-94ba-ec05b3bb5258 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1258 microversion: 2.1 time: 0.090947 Oct 04 08:55:57.547882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 388/844] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:57 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1258 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:55:58.571669 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:58.573203 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:58.580515 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:58.581069 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:58.581587 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:58.582492 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:58.582975 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:58.583391 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:58.676911 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:58.687608 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-0f6e149a-1f9a-42c1-89d8-b0b001983b22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1318 microversion: 2.1 time: 0.119148 Oct 04 08:55:58.688576 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 457/845] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:58 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1318 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:55:59.726929 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:55:59.729467 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:55:59.745565 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:59.749872 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:59.750596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:59.751734 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:55:59.756653 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:55:59.757363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:55:59.951046 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:55:59.961139 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a166a6e3-2d1b-40a5-909a-b48da617cc24 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.251667 Oct 04 08:55:59.962119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 389/846] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:55:59 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:00.993918 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:00.994799 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:01.004340 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:01.005110 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:01.005739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:01.006707 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:01.007348 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:01.007944 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:01.180082 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:01.189252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c1fd62b5-29d5-47e5-8d66-2cc4a04e61b0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.199177 Oct 04 08:56:01.203233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 458/847] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:00 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:02.214033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:02.214696 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:02.223921 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:02.224693 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:02.225331 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:02.226392 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:02.227118 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:02.238674 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:02.439133 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:02.451318 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a3123c5d-2ace-4fca-adb0-68e918affa94 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.241709 Oct 04 08:56:02.452492 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 390/848] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:02 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:03.479538 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:03.480831 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:03.490017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:03.491646 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:03.492393 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:03.493109 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:03.493760 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:03.494183 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:03.667333 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:03.675767 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-74c2ab1c-abc2-4b5b-a862-f6aac245d93b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.200186 Oct 04 08:56:03.676614 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 459/849] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:03 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:04.706123 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:04.708638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:04.727651 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:04.728257 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:04.728696 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:04.729542 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:04.730237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:04.730715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:04.929121 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:04.930610 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-db6cd394-4823-474e-8839-b675b379535d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.234146 Oct 04 08:56:04.943019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 391/850] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:04 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:05.959645 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:05.960830 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:05.971560 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:05.972042 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:05.972478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:05.973180 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:05.973692 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:05.974054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:06.224981 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:06.234997 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-bb862fbb-67f3-4433-91f0-c40141650a35 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.281146 Oct 04 08:56:06.235702 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 460/851] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:05 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:07.261398 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:07.262051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:07.269103 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:07.269581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:07.270025 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:07.270753 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:07.271204 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:07.271626 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:07.434778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:07.444764 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-607a742e-d20e-4d86-8c15-616709f4d71f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.187140 Oct 04 08:56:07.445519 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 392/852] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:07 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:08.421882 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-9e861615-44cd-4f4a-8bb8-f60ee7f04871 req-934accf7-4e36-4aad-a15a-4f83cd74180d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6b677e0-5baf-47a7-9a68-f18c5e915b55", "tag": "cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:08.428581 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=2 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:08.429245 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=6 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:08.434963 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e861615-44cd-4f4a-8bb8-f60ee7f04871 req-934accf7-4e36-4aad-a15a-4f83cd74180d service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:08.435454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e861615-44cd-4f4a-8bb8-f60ee7f04871 req-934accf7-4e36-4aad-a15a-4f83cd74180d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:08.435890 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-9e861615-44cd-4f4a-8bb8-f60ee7f04871 req-934accf7-4e36-4aad-a15a-4f83cd74180d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:08.458848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-9e861615-44cd-4f4a-8bb8-f60ee7f04871 req-934accf7-4e36-4aad-a15a-4f83cd74180d service nova] Creating event network-changed:cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 for instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 on hv-cinder-86020 Oct 04 08:56:08.468105 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:08.468771 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:08.471051 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=6 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:08.472522 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-9e861615-44cd-4f4a-8bb8-f60ee7f04871 req-934accf7-4e36-4aad-a15a-4f83cd74180d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054037 Oct 04 08:56:08.474017 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 461/853] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:56:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:56:08.478230 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:08.478867 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:08.479187 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:08.479904 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:08.480399 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:08.480827 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:08.712820 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:08.724596 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b8e96b57-7e8b-442d-972d-62f01ab103c4 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.260168 Oct 04 08:56:08.724860 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 393/854] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:08 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:09.749956 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:09.750956 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:09.758742 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:09.759244 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:09.760044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:09.763461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:09.763869 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:09.764308 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:09.911852 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:09.919823 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-2ff271a4-ab2e-4c14-8f42-f1aa4af8490f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.175192 Oct 04 08:56:09.921045 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 462/855] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:09 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:10.945184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:10.945794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:10.951946 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:10.952425 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:10.952854 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:10.953541 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:10.954117 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:10.954534 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:11.124756 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:11.134821 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a0cf023c-1e54-41bc-96da-80a4bbc877b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.193666 Oct 04 08:56:11.135452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 394/856] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:10 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:12.160112 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:12.160739 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:12.170525 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:12.171370 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:12.172046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:12.173487 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:12.174895 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:12.175502 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:12.343495 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:12.354046 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-65483dcf-e6fc-4c05-b9f5-5ed31ba692ab tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.199254 Oct 04 08:56:12.354668 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 463/857] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:12 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:13.379119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:13.379914 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:13.386988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:13.387483 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:13.387924 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:13.388638 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:13.389094 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:13.389519 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:13.537765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:13.552728 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-95a3c986-215a-43c6-8fb0-a52abd42cc22 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.178575 Oct 04 08:56:13.554496 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 395/858] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:13 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:14.432672 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-2bbc9833-9244-4344-b049-a979aa0b19af req-dff44d4f-ff84-4c34-b944-5abe05feca75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c6b677e0-5baf-47a7-9a68-f18c5e915b55", "name": "network-vif-plugged", "status": "completed", "tag": "cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:14.448705 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2bbc9833-9244-4344-b049-a979aa0b19af req-dff44d4f-ff84-4c34-b944-5abe05feca75 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:14.449416 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2bbc9833-9244-4344-b049-a979aa0b19af req-dff44d4f-ff84-4c34-b944-5abe05feca75 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:14.450051 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-2bbc9833-9244-4344-b049-a979aa0b19af req-dff44d4f-ff84-4c34-b944-5abe05feca75 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:14.470481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-2bbc9833-9244-4344-b049-a979aa0b19af req-dff44d4f-ff84-4c34-b944-5abe05feca75 service nova] Creating event network-vif-plugged:cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 for instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 on hv-cinder-86020 Oct 04 08:56:14.477885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-2bbc9833-9244-4344-b049-a979aa0b19af req-dff44d4f-ff84-4c34-b944-5abe05feca75 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050800 Oct 04 08:56:14.478710 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 464/859] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:56:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:56:14.588999 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:14.589687 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:14.598460 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:14.599116 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:14.599726 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:14.600665 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:14.601323 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:14.601867 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:14.823452 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:14.835955 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-13f4218a-04d4-4a9b-be8d-d7beda10da4c tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.250659 Oct 04 08:56:14.836912 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 396/860] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:14 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:15.863892 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:15.864603 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:15.871330 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:15.871820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:15.872261 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:15.872957 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:15.873409 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:15.873833 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:16.050496 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:16.059544 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-16ad6a9f-fe95-49ed-9795-882a74986577 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1358 microversion: 2.1 time: 0.201848 Oct 04 08:56:16.063775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 465/861] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:15 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1358 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:17.083442 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:17.084135 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:17.091996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.092635 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.093205 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.096132 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.096805 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.097395 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.247083 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:17.256257 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-91fab879-39b2-4dcf-a096-f15fa46e0936 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1547 microversion: 2.1 time: 0.176727 Oct 04 08:56:17.256856 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 397/862] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:17 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1547 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:17.298006 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:17.299900 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:17.310620 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.311340 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.312092 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.313835 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.314481 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.315253 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.482439 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:17.493242 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-40757427-8102-45dc-b216-139ff8da38d8 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1547 microversion: 2.1 time: 0.202557 Oct 04 08:56:17.493833 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 466/863] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:17 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1547 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:17.518273 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:17.525344 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:17.526308 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Searching by: {'deleted': False, 'project_id': '8a08fa28d7374a029062132a73bb8ab6'} {{(pid=127143) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 04 08:56:17.535091 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.535766 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.536333 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.537336 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.537898 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.538450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.642971 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.multi_cell_list [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=127143) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 04 08:56:17.756064 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-835e7dc0-9d1d-4442-966f-e024dd237fff tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.243791 Oct 04 08:56:17.756578 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 398/864] 10.222.0.16 () {58 vars in 1125 bytes} [Tue Oct 4 08:56:17 2022] GET /compute/v2.1/servers => generated 343 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:56:17.768257 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:17.768879 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:17.775121 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.775607 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.776062 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.776764 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:17.777202 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:17.777620 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:17.929791 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:17.939547 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d145892b-51ee-4149-bd2c-68d1000261ce tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1547 microversion: 2.1 time: 0.174779 Oct 04 08:56:17.940192 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 467/865] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:17 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1547 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:19.596282 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "604d2051-9c73-4552-bbb5-7828b3beaa30"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:19.598604 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:19.605667 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:19.606143 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:19.606599 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:19.607288 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:19.607732 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:19.608147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:19.917778 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-47234473-aeed-402d-a637-766c4cdd28da tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.325355 Oct 04 08:56:19.921807 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 399/866] 10.222.0.16 () {60 vars in 1323 bytes} [Tue Oct 4 08:56:19 2022] POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments => generated 194 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:56:31.867693 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:31.868300 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:31.870842 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=3 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:31.871529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=60 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:31.878097 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:31.878588 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:31.879016 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:31.879709 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:31.881326 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:31.881326 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:32.015929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:32.024533 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-d1997460-f1d9-4f29-8a45-67520da54775 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1593 microversion: 2.1 time: 0.160687 Oct 04 08:56:32.025103 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 468/867] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:31 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1593 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:34.044054 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:34.045759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:34.046541 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:34.052189 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-0755aed9-c441-4aba-9ce3-22cd71b164bf req-fe5576ee-d953-4a6b-aa5c-eb788220dac8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6b677e0-5baf-47a7-9a68-f18c5e915b55", "tag": "cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:34.082355 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-0755aed9-c441-4aba-9ce3-22cd71b164bf req-fe5576ee-d953-4a6b-aa5c-eb788220dac8 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:34.082887 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-0755aed9-c441-4aba-9ce3-22cd71b164bf req-fe5576ee-d953-4a6b-aa5c-eb788220dac8 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:34.083339 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-0755aed9-c441-4aba-9ce3-22cd71b164bf req-fe5576ee-d953-4a6b-aa5c-eb788220dac8 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:34.109280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-0755aed9-c441-4aba-9ce3-22cd71b164bf req-fe5576ee-d953-4a6b-aa5c-eb788220dac8 service nova] Creating event network-changed:cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 for instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 on hv-cinder-86020 Oct 04 08:56:34.122720 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-0755aed9-c441-4aba-9ce3-22cd71b164bf req-fe5576ee-d953-4a6b-aa5c-eb788220dac8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079993 Oct 04 08:56:34.123711 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 400/868] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:56:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:56:35.230871 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-964438823"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:35.231557 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:35.239490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:35.239980 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:35.240415 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:35.241186 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:35.241641 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:35.242061 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:35.386334 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.network.security_group_api [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Adding security group 1345b8bf-d982-4a7d-9f92-2b70c989d0e3 to port cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 Oct 04 08:56:35.699068 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-90470d88-9e68-467d-8a9a-f8847fb64613 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/action" status: 202 len: 0 microversion: 2.1 time: 0.472003 Oct 04 08:56:35.699606 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 469/869] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:56:35 2022] POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/action => generated 0 bytes in 473 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:56:35.709374 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:35.710251 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:35.719244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:35.719737 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:35.720193 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:35.723184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:35.723668 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:35.724137 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:35.888833 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:35.900019 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-19a5d9b3-d3f4-42ac-a779-9ab12474921b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1757 microversion: 2.1 time: 0.195886 Oct 04 08:56:35.900796 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 401/870] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:35 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1757 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:36.140783 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-1ed52611-6cdd-40eb-a7d6-65ff35185d94 req-d98c4fde-5fc7-4f74-88db-80d0beb71a33 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6b677e0-5baf-47a7-9a68-f18c5e915b55", "tag": "cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:36.152916 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-1ed52611-6cdd-40eb-a7d6-65ff35185d94 req-d98c4fde-5fc7-4f74-88db-80d0beb71a33 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:36.153848 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-1ed52611-6cdd-40eb-a7d6-65ff35185d94 req-d98c4fde-5fc7-4f74-88db-80d0beb71a33 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:36.154280 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-1ed52611-6cdd-40eb-a7d6-65ff35185d94 req-d98c4fde-5fc7-4f74-88db-80d0beb71a33 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:36.170520 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-1ed52611-6cdd-40eb-a7d6-65ff35185d94 req-d98c4fde-5fc7-4f74-88db-80d0beb71a33 service nova] Creating event network-changed:cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 for instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 on hv-cinder-86020 Oct 04 08:56:36.176929 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-1ed52611-6cdd-40eb-a7d6-65ff35185d94 req-d98c4fde-5fc7-4f74-88db-80d0beb71a33 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040733 Oct 04 08:56:36.177790 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 470/871] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:56:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:56:39.149286 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1665346068", "imageRef": "b2d50eae-6db7-4be8-ab9b-7d0ad42807a0", "flavorRef": "452"}} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:39.203708 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.network.neutron [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] validate_networks() for None {{(pid=127143) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 04 08:56:39.483666 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Flavor limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 04 08:56:39.484113 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Image limits 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 04 08:56:39.484591 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Flavor pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 04 08:56:39.484996 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Image pref 0:0:0 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 04 08:56:39.485407 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.virt.hardware [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127143) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 04 08:56:39.494950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:39.495430 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:39.495861 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:39.496583 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:39.497033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:39.497450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:39.514597 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Getting quotas for project 756737695aa644ff8a9ec362b654cf3f. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 04 08:56:39.518566 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.quota [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Getting quotas for user 7ef96a2e81f2404f960b8882659da9c4 and project 756737695aa644ff8a9ec362b654cf3f. Resources: {'cores', 'ram', 'instances'} {{(pid=127143) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 04 08:56:39.531900 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Going to run 1 instances... {{(pid=127143) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 04 08:56:39.533863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b2d50eae-6db7-4be8-ab9b-7d0ad42807a0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127143) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 04 08:56:39.584327 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0716cdd4-bd71-40ab-a656-0f674f7c77ff tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.441808 Oct 04 08:56:39.584993 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 402/872] 10.222.0.16 () {60 vars in 1147 bytes} [Tue Oct 4 08:56:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 443 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 04 08:56:39.598077 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b1750647-34ab-41fd-9ba0-33d8f223a973 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:39.599390 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b1750647-34ab-41fd-9ba0-33d8f223a973 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:39.652956 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b1750647-34ab-41fd-9ba0-33d8f223a973 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:39.661357 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b1750647-34ab-41fd-9ba0-33d8f223a973 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1215 microversion: 2.1 time: 0.068044 Oct 04 08:56:39.661927 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 471/873] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:39 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1215 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:40.683201 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-9f8dacff-a930-4ae9-a395-8d4b2301b792 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:40.683815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-9f8dacff-a930-4ae9-a395-8d4b2301b792 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:40.740846 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-9f8dacff-a930-4ae9-a395-8d4b2301b792 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:40.747696 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-9f8dacff-a930-4ae9-a395-8d4b2301b792 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1215 microversion: 2.1 time: 0.068377 Oct 04 08:56:40.748258 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 403/874] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:40 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1215 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:41.770975 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-35cecf8c-effd-4782-825b-013f2fee504b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:41.771546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-35cecf8c-effd-4782-825b-013f2fee504b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:41.840769 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-35cecf8c-effd-4782-825b-013f2fee504b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:41.849149 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-35cecf8c-effd-4782-825b-013f2fee504b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1215 microversion: 2.1 time: 0.081967 Oct 04 08:56:41.850144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 472/875] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:41 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1215 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:42.874025 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:42.874743 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:42.881622 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:42.882153 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:42.882722 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:42.883419 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:42.883870 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:42.884287 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:42.952885 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:42.961663 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-993bd86d-4edb-4259-8de1-9f5572e78257 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1273 microversion: 2.1 time: 0.091425 Oct 04 08:56:42.962237 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 404/876] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:42 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1273 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:43.986893 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:43.987521 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:43.995071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:43.995565 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:43.996009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:43.996952 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:43.997420 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:43.997844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:44.148543 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:44.163426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e64c0656-ecb8-475d-b4f2-d884abb2eaf8 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.181140 Oct 04 08:56:44.164319 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 473/877] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:43 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:44.465427 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-3a16426e-fdbd-45d0-9a59-923bce34b38b req-fa69f28a-4593-4d31-ae13-3431e738613f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38486ac9-5ec2-4af0-abc1-2d7df674b060", "tag": "ef8e9a4f-79e9-4a30-b22f-968ef4c3bef3"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:44.479878 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-3a16426e-fdbd-45d0-9a59-923bce34b38b req-fa69f28a-4593-4d31-ae13-3431e738613f service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:44.480282 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-3a16426e-fdbd-45d0-9a59-923bce34b38b req-fa69f28a-4593-4d31-ae13-3431e738613f service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:44.480779 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-3a16426e-fdbd-45d0-9a59-923bce34b38b req-fa69f28a-4593-4d31-ae13-3431e738613f service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:44.495152 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-3a16426e-fdbd-45d0-9a59-923bce34b38b req-fa69f28a-4593-4d31-ae13-3431e738613f service nova] Creating event network-changed:ef8e9a4f-79e9-4a30-b22f-968ef4c3bef3 for instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 on hv-cinder-86020 Oct 04 08:56:44.500599 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-3a16426e-fdbd-45d0-9a59-923bce34b38b req-fa69f28a-4593-4d31-ae13-3431e738613f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038909 Oct 04 08:56:44.501214 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 405/878] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:56:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:56:45.187352 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:45.187999 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:45.194885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:45.195380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:45.195822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:45.196579 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:45.196996 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:45.197559 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:45.331586 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:45.339998 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-04f33cbf-19cd-4d01-be84-53f3f09374ac tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.156591 Oct 04 08:56:45.340567 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 474/879] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:45 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:46.364186 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:46.364837 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:46.371542 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:46.372016 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:46.372450 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:46.373133 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:46.373581 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:46.373998 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:46.503519 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:46.512140 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-15b2d44e-f620-4d61-9d83-85dee38c9b66 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.153042 Oct 04 08:56:46.512715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 406/880] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:46 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:47.536549 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:47.537145 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:47.543471 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:47.543945 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:47.544380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:47.545068 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:47.545517 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:47.545934 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:47.683620 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:47.691253 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-66423c71-8c82-42a6-850a-ccbb98bf7c60 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.159310 Oct 04 08:56:47.692693 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 475/881] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:47 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:48.715649 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:48.716264 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:48.723018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:48.723504 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:48.724006 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:48.724700 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:48.725148 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:48.725561 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:48.875944 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:48.884568 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-c3e04725-1631-42d3-a51e-c14d34100106 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.172784 Oct 04 08:56:48.885151 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 407/882] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:48 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:49.913081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:49.913959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:49.920529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:49.921009 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:49.921432 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:49.922123 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:49.922584 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:49.923001 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:50.053673 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:50.061431 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-046fe8db-dde5-474c-a120-515efaf73bc0 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.152250 Oct 04 08:56:50.061997 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 476/883] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:49 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:50.667532 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-a938a69f-2573-49a6-87a2-db111c6c885c req-f8d08404-835d-49dc-adce-4602f2f0319a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38486ac9-5ec2-4af0-abc1-2d7df674b060", "name": "network-vif-plugged", "status": "completed", "tag": "ef8e9a4f-79e9-4a30-b22f-968ef4c3bef3"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:50.673256 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:50.680725 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-a938a69f-2573-49a6-87a2-db111c6c885c req-f8d08404-835d-49dc-adce-4602f2f0319a service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:50.681317 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-a938a69f-2573-49a6-87a2-db111c6c885c req-f8d08404-835d-49dc-adce-4602f2f0319a service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:50.681911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-a938a69f-2573-49a6-87a2-db111c6c885c req-f8d08404-835d-49dc-adce-4602f2f0319a service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:50.694353 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-a938a69f-2573-49a6-87a2-db111c6c885c req-f8d08404-835d-49dc-adce-4602f2f0319a service nova] Creating event network-vif-plugged:ef8e9a4f-79e9-4a30-b22f-968ef4c3bef3 for instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 on hv-cinder-86020 Oct 04 08:56:50.699435 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-a938a69f-2573-49a6-87a2-db111c6c885c req-f8d08404-835d-49dc-adce-4602f2f0319a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035442 Oct 04 08:56:50.700207 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 408/884] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:56:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:56:50.902090 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:50.904380 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:50.910925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:50.911641 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:50.912190 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:50.912891 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:50.913343 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:50.913914 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:50.993211 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-e34fe53a-ecd1-4499-b420-9217289f5f32 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/action" status: 202 len: 0 microversion: 2.1 time: 0.094564 Oct 04 08:56:50.994295 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 477/885] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:56:50 2022] POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/action => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:56:51.000315 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell0:SELECT=8 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:51.006092 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:51.006852 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:51.014223 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:51.014836 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:51.015280 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:51.015988 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:51.016443 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:51.016868 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:51.085750 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:51.086454 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:51.092836 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:51.093345 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:51.093775 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:51.094490 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:51.094938 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:51.095376 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:51.173518 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:51.182911 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-1a9b7bf2-8afe-4a6b-9b70-7e4a3ecea9ca tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1749 microversion: 2.1 time: 0.183386 Oct 04 08:56:51.183608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 409/886] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:50 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1749 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:51.254779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:51.262621 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-8b0cdb49-0d80-43f4-92a1-f10a3d0fef78 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.182321 Oct 04 08:56:51.263205 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 478/887] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:51 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:56:52.208853 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:52.209408 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:52.215903 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:52.216384 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:52.216816 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:52.217505 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:52.217950 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:52.218370 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:52.287306 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:52.287925 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:52.289885 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell0:SELECT=4 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:52.296143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:52.296945 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:52.297445 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:52.298220 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:52.298800 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:52.299289 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:52.378038 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:52.386639 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ffa50341-a7c2-4575-b3c3-b8a9ace9778b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1749 microversion: 2.1 time: 0.182893 Oct 04 08:56:52.389863 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 410/888] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:52 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1749 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:52.463461 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:52.471282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-55861e3d-c8e6-4831-90b1-1c2cc1cffa1d tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1315 microversion: 2.1 time: 0.188613 Oct 04 08:56:52.471858 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 479/889] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:52 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1315 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:56:53.412327 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:53.412885 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:53.420670 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.424952 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.425360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:53.427117 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.427917 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.428348 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:53.495601 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:53.496360 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:53.503983 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.504473 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.504902 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:53.505603 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.506054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.506468 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:53.601433 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:53.610976 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-52c6b9b9-44aa-4fb4-aa2e-fb3355c5760b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.203823 Oct 04 08:56:53.611574 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 411/890] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:53 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:53.663392 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:53.671572 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-a583019a-1637-49ec-988a-cbd9d75fad39 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1465 microversion: 2.1 time: 0.180051 Oct 04 08:56:53.672143 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 480/891] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:53 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1465 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:56:53.721769 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:53.722357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:53.733540 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.734281 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.735119 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:53.735973 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.736405 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.737219 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:53.771653 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:56:53.877192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-06d2425e-9793-40c5-a25a-a426b72654ef tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "DELETE /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 204 len: 0 microversion: 2.1 time: 0.158908 Oct 04 08:56:53.878202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 412/892] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:56:53 2022] DELETE /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:56:53.887772 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:53.888528 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:53.897718 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.898204 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.898662 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:53.899359 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:53.900015 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:53.900431 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:54.048386 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:54.056968 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-be4ec9fd-b172-4d4c-aa34-d3aa7c0a88b6 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1471 microversion: 2.1 time: 0.174042 Oct 04 08:56:54.057546 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 481/893] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:53 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1471 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:56:54.639590 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:54.640279 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:54.646765 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:54.647184 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:54.647617 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:54.648299 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:54.648742 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:54.649159 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:54.800073 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:54.809255 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5814d3a8-3755-40e2-89ff-faa4efc5dcba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.174150 Oct 04 08:56:54.809892 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 413/894] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:54 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:55.082163 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:55.083065 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:55.089180 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:55.089658 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:55.090091 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:55.090802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:55.091255 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:55.091671 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:55.225033 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:55.232991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cda3b85c-fba2-4be8-ad1a-f94f72ee3a6b tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1471 microversion: 2.1 time: 0.155909 Oct 04 08:56:55.233568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 482/895] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:55 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1471 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:55.780809 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-f1d68bb0-85fb-4c7d-9604-18a49c89671a req-e5eadbe5-1bb0-4bfb-a683-81e36f5a46d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38486ac9-5ec2-4af0-abc1-2d7df674b060", "name": "network-vif-deleted", "tag": "ef8e9a4f-79e9-4a30-b22f-968ef4c3bef3"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:56:55.802304 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f1d68bb0-85fb-4c7d-9604-18a49c89671a req-e5eadbe5-1bb0-4bfb-a683-81e36f5a46d8 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:55.802822 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f1d68bb0-85fb-4c7d-9604-18a49c89671a req-e5eadbe5-1bb0-4bfb-a683-81e36f5a46d8 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:55.803963 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-f1d68bb0-85fb-4c7d-9604-18a49c89671a req-e5eadbe5-1bb0-4bfb-a683-81e36f5a46d8 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:55.815724 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-f1d68bb0-85fb-4c7d-9604-18a49c89671a req-e5eadbe5-1bb0-4bfb-a683-81e36f5a46d8 service nova] Creating event network-vif-deleted:ef8e9a4f-79e9-4a30-b22f-968ef4c3bef3 for instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 on hv-cinder-86020 Oct 04 08:56:55.822444 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-f1d68bb0-85fb-4c7d-9604-18a49c89671a req-e5eadbe5-1bb0-4bfb-a683-81e36f5a46d8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045664 Oct 04 08:56:55.823429 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 414/896] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:56:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 04 08:56:55.833213 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:55.833840 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:55.840191 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:55.840762 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:55.841273 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:55.842290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:55.842909 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:55.843326 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:56.010585 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:56.025991 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-283ff1dc-1cad-467b-9922-7f02abdd59b9 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.197507 Oct 04 08:56:56.026627 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 483/897] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:55 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:56.260469 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:56.261206 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:56.268549 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:56.269028 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:56.270052 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:56.270859 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:56.271309 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:56.271726 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:56.306147 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:56.306714 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:56.307109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:56.308040 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lazy-loading 'fault' on Instance uuid 38486ac9-5ec2-4af0-abc1-2d7df674b060 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:56:56.352361 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ef96a2e81f2404f960b8882659da9c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '756737695aa644ff8a9ec362b654cf3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:56.359387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ce559d4b-a3bf-43e8-ae47-ca5efacc4d86 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 200 len: 1305 microversion: 2.1 time: 0.104861 Oct 04 08:56:56.360393 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 415/898] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:56 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 1305 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:57.049480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:57.050101 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:57.056749 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:57.057224 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:57.057659 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:57.058344 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:57.058820 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:57.059241 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:57.202129 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:57.211491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-25d202ad-e1f8-4dee-aeca-192062097d9d tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.165854 Oct 04 08:56:57.212065 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 484/899] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:57 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:57.385221 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:57.385830 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] [instance: 38486ac9-5ec2-4af0-abc1-2d7df674b060] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:57.392277 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:57.392762 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:57.393192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:57.393885 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:57.394401 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:57.394875 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:57.408268 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.wsgi [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] HTTP exception thrown: Instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 could not be found. Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Request method failure captured: Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: request: GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 HTTP/1.1 Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept: application/json Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Accept-Encoding: identity Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Connection: close Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Length: 0 Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Content-Type: application/json Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Host: 10.222.0.16 Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: Openstack-System-Scope: None Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: User-Agent: python-urllib3/1.26.12 Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Auth-Token: gAAAAABjO_VFvPpyKieFzjk0tSy_d5ncMz_LeI8uzQ53liwx3YLpzZSlhSDTfFwjhKAvpQ89KkaFYIX6HaTE-sdpYaQb5rrsu1ygVhryCEpk14uhI27DXkWsAR7O_0L0PnhA-LWevDFukq74ci5-E5kXlsU29G3V4hqvUY16potgPNCdTO0uar8 Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Id: None Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Domain-Name: None Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Identity-Status: Confirmed Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Is-Admin-Project: True Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Id: default Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Domain-Name: Default Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Id: 756737695aa644ff8a9ec362b654cf3f Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Project-Name: tempest-VolumesNegativeTest-2109517988 Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Role: member,reader Oct 04 08:56:57.410936 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Roles: member,reader Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/756737695aa644ff8a9ec362b654cf3f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/756737695aa644ff8a9ec362b654cf3f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/756737695aa644ff8a9ec362b654cf3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_756737695aa644ff8a9ec362b654cf3f", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant: tempest-VolumesNegativeTest-2109517988 Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Id: 756737695aa644ff8a9ec362b654cf3f Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-Tenant-Name: tempest-VolumesNegativeTest-2109517988 Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User: tempest-VolumesNegativeTest-2109517988-project Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Id: default Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Domain-Name: Default Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Id: 7ef96a2e81f2404f960b8882659da9c4 Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: X-User-Name: tempest-VolumesNegativeTest-2109517988-project Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: method: > Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: exception: Instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 could not be found. Oct 04 08:56:57.412697 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: action_args: {'id': '38486ac9-5ec2-4af0-abc1-2d7df674b060'} Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 could not be found. Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:56:57.413791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 could not be found. Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 could not be found. Oct 04 08:56:57.415480 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: ERROR nova.api.openstack.wsgi Oct 04 08:56:57.421240 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] Returning 404 to user: Instance 38486ac9-5ec2-4af0-abc1-2d7df674b060 could not be found. {{(pid=127143) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:56:57.421240 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-5450aa57-c410-44b7-8b0e-8c76662c72a4 tempest-VolumesNegativeTest-2109517988 tempest-VolumesNegativeTest-2109517988-project] 10.222.0.16 "GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060" status: 404 len: 111 microversion: 2.1 time: 0.031069 Oct 04 08:56:57.425033 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 416/900] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:57 2022] GET /compute/v2.1/servers/38486ac9-5ec2-4af0-abc1-2d7df674b060 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:56:58.234841 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:58.235463 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:58.242514 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:58.243043 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:58.243480 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:58.244178 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:58.244655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:58.245052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:58.401177 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:58.410207 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-cd812263-605f-42b2-8299-f29205c8e47f tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.179327 Oct 04 08:56:58.411099 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 485/901] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:58 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:56:59.433900 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:56:59.434485 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:56:59.441089 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:59.441576 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:59.442636 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:59.443360 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:56:59.443815 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:56:59.444239 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:56:59.591020 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:56:59.599789 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-ae96ab77-57a2-450d-a1a3-ec3e043f42e3 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.169823 Oct 04 08:56:59.600357 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 417/902] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:56:59 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:57:00.624779 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:00.625334 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:00.633054 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:00.633538 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:00.633974 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:00.634696 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:00.635154 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:00.635576 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:00.654073 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:00.784457 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:57:00.793060 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-7b91c27a-21b0-4259-8271-50d9c9a2e44a tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.172237 Oct 04 08:57:00.793678 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 486/903] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:00 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:57:01.818263 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:01.819007 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:01.825834 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:01.826322 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:01.826794 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:01.827507 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:01.827957 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:01.828376 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:01.985933 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:57:01.995208 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-b42e6b77-37f5-42bc-989e-b81ba37ce460 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.181711 Oct 04 08:57:01.995942 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 418/904] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:01 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:57:03.022714 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:03.023322 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:03.027081 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=43 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:03.036692 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:03.037191 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:03.037626 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:03.038332 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:03.038810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:03.039233 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:03.186951 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:57:03.195239 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-b131eb29-709d-4f40-8694-660d84b1acdd tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1754 microversion: 2.1 time: 0.178817 Oct 04 08:57:03.195811 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 487/905] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:03 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1754 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 04 08:57:04.218363 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:04.219018 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:04.226300 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:04.226827 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:04.227266 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:04.228001 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:04.228454 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:04.228882 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:04.248004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:04.386463 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:57:04.394975 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0422e97e-f84c-4711-b7e7-15ba7381497b tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1757 microversion: 2.1 time: 0.180356 Oct 04 08:57:04.395563 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 419/906] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:04 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1757 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:57:36.855023 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-964438823"}} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:57:36.855681 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:36.858071 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 04 08:57:36.858210 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Tue Oct 4 08:57:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/action (ip 10.222.0.16) !!! Oct 04 08:57:36.859984 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_api:SELECT=1 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:36.860711 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG dbcounter [-] [127142] Writing DB stats nova_cell1:SELECT=8 {{(pid=127142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:36.874216 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:36.874748 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:36.875185 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:36.875897 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:36.876352 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:36.876778 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:37.010144 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.network.security_group_api [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Removing security group 1345b8bf-d982-4a7d-9f92-2b70c989d0e3 from port cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 Oct 04 08:57:37.312252 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-67ebdfce-a030-4944-a605-b39f727f719e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/action" status: 202 len: 0 microversion: 2.1 time: 0.460918 Oct 04 08:57:37.316751 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 488/907] 10.222.0.16 () {60 vars in 1278 bytes} [Tue Oct 4 08:57:36 2022] POST /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/action => generated 0 bytes in 463 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:57:37.323420 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [req-4675238a-6698-40d3-8b17-df811242fdd7 req-953a806a-1875-45f5-8027-902bce1bab69 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6b677e0-5baf-47a7-9a68-f18c5e915b55", "tag": "cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7"}]} {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:57:37.333464 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_api:SELECT=60,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:37.335355 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG dbcounter [-] [127143] Writing DB stats nova_cell1:SELECT=2 {{(pid=127143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:37.340678 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-4675238a-6698-40d3-8b17-df811242fdd7 req-953a806a-1875-45f5-8027-902bce1bab69 service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:37.341871 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-4675238a-6698-40d3-8b17-df811242fdd7 req-953a806a-1875-45f5-8027-902bce1bab69 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:37.343759 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [req-4675238a-6698-40d3-8b17-df811242fdd7 req-953a806a-1875-45f5-8027-902bce1bab69 service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:37.358587 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.compute.server_external_events [req-4675238a-6698-40d3-8b17-df811242fdd7 req-953a806a-1875-45f5-8027-902bce1bab69 service nova] Creating event network-changed:cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 for instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 on hv-cinder-86020 Oct 04 08:57:37.371522 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [req-4675238a-6698-40d3-8b17-df811242fdd7 req-953a806a-1875-45f5-8027-902bce1bab69 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051750 Oct 04 08:57:37.375109 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 420/908] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:57:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:57:38.703582 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:38.704147 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:38.710231 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:38.710754 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:38.711185 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:38.711877 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:38.712321 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:38.712735 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:38.802960 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.objects.instance [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lazy-loading 'services' on Instance uuid c6b677e0-5baf-47a7-9a68-f18c5e915b55 {{(pid=127142) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:57:38.878833 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-febf00e1-4b60-45db-9af6-08e5733011c0 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments/604d2051-9c73-4552-bbb5-7828b3beaa30" status: 202 len: 0 microversion: 2.1 time: 0.179004 Oct 04 08:57:38.879655 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 489/909] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:57:38 2022] DELETE /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments/604d2051-9c73-4552-bbb5-7828b3beaa30 => generated 0 bytes in 181 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 04 08:57:38.889924 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:38.892902 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:38.901273 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:38.901940 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:38.902436 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:38.903221 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:38.903745 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:38.904862 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:38.943822 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-0e4b70b3-c864-4f7b-873d-e18dfbdd21e2 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058920 Oct 04 08:57:38.944755 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 421/910] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:57:38 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:57:39.398959 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-b6e6304c-e5e4-4862-92b9-daf45fa18fa8 req-3347d8d8-8562-477e-b677-7a6ad8882a4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c6b677e0-5baf-47a7-9a68-f18c5e915b55", "tag": "cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:57:39.410426 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-b6e6304c-e5e4-4862-92b9-daf45fa18fa8 req-3347d8d8-8562-477e-b677-7a6ad8882a4e service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:39.410967 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-b6e6304c-e5e4-4862-92b9-daf45fa18fa8 req-3347d8d8-8562-477e-b677-7a6ad8882a4e service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:39.411484 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-b6e6304c-e5e4-4862-92b9-daf45fa18fa8 req-3347d8d8-8562-477e-b677-7a6ad8882a4e service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:39.423024 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-b6e6304c-e5e4-4862-92b9-daf45fa18fa8 req-3347d8d8-8562-477e-b677-7a6ad8882a4e service nova] Creating event network-changed:cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 for instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 on hv-cinder-86020 Oct 04 08:57:39.428343 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-b6e6304c-e5e4-4862-92b9-daf45fa18fa8 req-3347d8d8-8562-477e-b677-7a6ad8882a4e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033580 Oct 04 08:57:39.429057 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 490/911] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:57:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:57:39.957934 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:39.960122 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:39.967037 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:39.967521 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:39.968632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:39.969337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:39.969791 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:39.970325 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:40.001282 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-bd3c78e4-e7a6-4d75-9a65-2ad48c6b7e30 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048360 Oct 04 08:57:40.002260 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 422/912] 10.222.0.16 () {58 vars in 1302 bytes} [Tue Oct 4 08:57:39 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 04 08:57:40.013802 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:40.014377 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:40.022290 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:40.022822 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:40.023258 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:40.023990 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:40.024441 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:40.024863 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:40.101055 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] HTTP exception thrown: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 is not attached to volume 604d2051-9c73-4552-bbb5-7828b3beaa30 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Request method failure captured: Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: DELETE /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments/604d2051-9c73-4552-bbb5-7828b3beaa30 HTTP/1.1 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_UKb5h9R65sRMs6ldX_4T_rfzfSgeI4bOjAbcmjdvGM8AQsPlav-OFVIYQ_WqlA7wJuIV7lc0mOAj4pYTxbJUz2881642xyPLtSUORAJME3UMHhZck4b0zv4L9Q0xR8I7lyNyUViVF-vnlA0Xa4bcM20vqQmX9ngpXiYX9slggjYP4 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 8a08fa28d7374a029062132a73bb8ab6 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-TestMinimumBasicScenario-1330805197 Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:57:40.103491 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8a08fa28d7374a029062132a73bb8ab6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8a08fa28d7374a029062132a73bb8ab6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8a08fa28d7374a029062132a73bb8ab6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8a08fa28d7374a029062132a73bb8ab6", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-TestMinimumBasicScenario-1330805197 Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 8a08fa28d7374a029062132a73bb8ab6 Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1330805197 Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-TestMinimumBasicScenario-1330805197-project Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: b62e31d5979c426fb5b6416f087692ff Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-TestMinimumBasicScenario-1330805197-project Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:57:40.105478 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 is not attached to volume 604d2051-9c73-4552-bbb5-7828b3beaa30 Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'server_id': 'c6b677e0-5baf-47a7-9a68-f18c5e915b55', 'id': '604d2051-9c73-4552-bbb5-7828b3beaa30'} Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 604d2051-9c73-4552-bbb5-7828b3beaa30. Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:57:40.108044 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 is not attached to volume 604d2051-9c73-4552-bbb5-7828b3beaa30 Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 is not attached to volume 604d2051-9c73-4552-bbb5-7828b3beaa30 Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:40.109555 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Returning 404 to user: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 is not attached to volume 604d2051-9c73-4552-bbb5-7828b3beaa30 {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:57:40.110876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-da1a3ec9-90bf-46f6-9b0e-b5743d2ef5d6 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments/604d2051-9c73-4552-bbb5-7828b3beaa30" status: 404 len: 154 microversion: 2.1 time: 0.096426 Oct 04 08:57:40.110876 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 491/913] 10.222.0.16 () {58 vars in 1416 bytes} [Tue Oct 4 08:57:40 2022] DELETE /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55/os-volume_attachments/604d2051-9c73-4552-bbb5-7828b3beaa30 => generated 154 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:57:41.253981 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:41.254495 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:41.260571 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:41.261046 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:41.261473 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:41.262150 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:41.262632 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:41.263055 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:41.287458 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Going to try to terminate instance {{(pid=127143) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 04 08:57:41.361488 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-a58a611b-85d1-4dd6-8ac6-8eab3230e2bf tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 204 len: 0 microversion: 2.1 time: 0.110990 Oct 04 08:57:41.362262 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 423/914] 10.222.0.16 () {58 vars in 1239 bytes} [Tue Oct 4 08:57:41 2022] DELETE /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 04 08:57:41.372371 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:41.372988 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:41.379394 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:41.379878 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:41.380322 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:41.381282 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:41.381740 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:41.382169 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:41.519810 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.policy [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127142) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:57:41.527858 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-c95d3702-e2ba-401c-9d15-f9949f6cba82 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1553 microversion: 2.1 time: 0.160386 Oct 04 08:57:41.528415 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 492/915] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:41 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1553 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:57:42.552192 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:42.553004 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:42.559337 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:42.559814 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:42.560244 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:42.560924 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:42.561369 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:42.561787 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:42.691702 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:57:42.700372 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-af899da9-44c2-4829-8820-00edd72942ba tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1553 microversion: 2.1 time: 0.152391 Oct 04 08:57:42.700948 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 424/916] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:42 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1553 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:57:43.540905 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [req-ffa997e2-184f-46a4-b384-fda6fa20a9dd req-2eb13eae-93da-492c-8a39-dd76f340cb4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c6b677e0-5baf-47a7-9a68-f18c5e915b55", "name": "network-vif-deleted", "tag": "cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7"}]} {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 04 08:57:43.554311 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-ffa997e2-184f-46a4-b384-fda6fa20a9dd req-2eb13eae-93da-492c-8a39-dd76f340cb4d service nova] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:43.554809 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-ffa997e2-184f-46a4-b384-fda6fa20a9dd req-2eb13eae-93da-492c-8a39-dd76f340cb4d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:43.555230 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [req-ffa997e2-184f-46a4-b384-fda6fa20a9dd req-2eb13eae-93da-492c-8a39-dd76f340cb4d service nova] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:43.569075 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.compute.server_external_events [req-ffa997e2-184f-46a4-b384-fda6fa20a9dd req-2eb13eae-93da-492c-8a39-dd76f340cb4d service nova] Creating event network-vif-deleted:cab159b2-e7d2-4fa4-a60a-ccbffbb1d5f7 for instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 on hv-cinder-86020 Oct 04 08:57:43.575374 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [req-ffa997e2-184f-46a4-b384-fda6fa20a9dd req-2eb13eae-93da-492c-8a39-dd76f340cb4d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039292 Oct 04 08:57:43.575977 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 493/917] 10.222.0.16 () {64 vars in 1317 bytes} [Tue Oct 4 08:57:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 04 08:57:43.724938 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:43.726075 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.compute.api [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127143) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:43.732688 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:43.733174 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:43.733608 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:43.734294 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:43.734780 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:43.735202 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:43.766881 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:43.767283 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:43.767715 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG oslo_concurrency.lockutils [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127143) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:43.768162 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.objects.instance [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lazy-loading 'fault' on Instance uuid c6b677e0-5baf-47a7-9a68-f18c5e915b55 {{(pid=127143) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 04 08:57:43.810706 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.policy [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b62e31d5979c426fb5b6416f087692ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8a08fa28d7374a029062132a73bb8ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127143) authorize /opt/stack/nova/nova/policy.py:199}} Oct 04 08:57:43.818979 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-07d12e06-04e1-49ba-9f9a-e8c378322e50 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 200 len: 1348 microversion: 2.1 time: 0.098306 Oct 04 08:57:43.819564 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 425/918] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:43 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 1348 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 04 08:57:44.843568 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '>' {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:44.844264 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.compute.api [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] [instance: c6b677e0-5baf-47a7-9a68-f18c5e915b55] Fetching instance by UUID {{(pid=127142) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 04 08:57:44.850941 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:44.851412 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:44.851844 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:44.852529 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Acquiring lock "4bcedf19-741d-498b-8c07-5371abba3edf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 04 08:57:44.852972 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 04 08:57:44.853387 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG oslo_concurrency.lockutils [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Lock "4bcedf19-741d-498b-8c07-5371abba3edf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127142) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 04 08:57:44.867403 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.wsgi [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] HTTP exception thrown: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 could not be found. Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Request method failure captured: Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: request: GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 HTTP/1.1 Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept: application/json Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Accept-Encoding: identity Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Connection: close Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Length: 0 Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Content-Type: application/json Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Host: 10.222.0.16 Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: Openstack-System-Scope: None Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: User-Agent: python-urllib3/1.26.12 Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Auth-Token: gAAAAABjO_UKb5h9R65sRMs6ldX_4T_rfzfSgeI4bOjAbcmjdvGM8AQsPlav-OFVIYQ_WqlA7wJuIV7lc0mOAj4pYTxbJUz2881642xyPLtSUORAJME3UMHhZck4b0zv4L9Q0xR8I7lyNyUViVF-vnlA0Xa4bcM20vqQmX9ngpXiYX9slggjYP4 Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Id: None Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Domain-Name: None Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Identity-Status: Confirmed Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Is-Admin-Project: True Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Id: default Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Domain-Name: Default Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Id: 8a08fa28d7374a029062132a73bb8ab6 Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Project-Name: tempest-TestMinimumBasicScenario-1330805197 Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Role: member,reader Oct 04 08:57:44.869052 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Roles: member,reader Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8a08fa28d7374a029062132a73bb8ab6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8a08fa28d7374a029062132a73bb8ab6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8a08fa28d7374a029062132a73bb8ab6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8a08fa28d7374a029062132a73bb8ab6", "adminURL": "http://10.222.0.16:8080"}]}] Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant: tempest-TestMinimumBasicScenario-1330805197 Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Id: 8a08fa28d7374a029062132a73bb8ab6 Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1330805197 Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User: tempest-TestMinimumBasicScenario-1330805197-project Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Id: default Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Domain-Name: Default Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Id: b62e31d5979c426fb5b6416f087692ff Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: X-User-Name: tempest-TestMinimumBasicScenario-1330805197-project Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: method: > Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: exception: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 could not be found. Oct 04 08:57:44.870770 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: action_args: {'id': 'c6b677e0-5baf-47a7-9a68-f18c5e915b55'} Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: {{(pid=127142) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 could not be found. Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 04 08:57:44.871789 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 could not be found. Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 could not be found. Oct 04 08:57:44.873388 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: ERROR nova.api.openstack.wsgi Oct 04 08:57:44.874910 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: DEBUG nova.api.openstack.wsgi [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Returning 404 to user: Instance c6b677e0-5baf-47a7-9a68-f18c5e915b55 could not be found. {{(pid=127142) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 04 08:57:44.874910 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: INFO nova.api.openstack.requestlog [None req-facfd2a1-6e87-4d7c-a9d1-c754dca7f482 tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55" status: 404 len: 111 microversion: 2.1 time: 0.031086 Oct 04 08:57:44.874910 dv-cinder-860209-1-iscsi devstack@n-api.service[127142]: [pid: 127142|app: 0|req: 494/919] 10.222.0.16 () {58 vars in 1236 bytes} [Tue Oct 4 08:57:44 2022] GET /compute/v2.1/servers/c6b677e0-5baf-47a7-9a68-f18c5e915b55 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 04 08:57:44.881716 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: DEBUG nova.api.openstack.wsgi [None req-56a21f0c-ae24-40e1-b457-b84fa2c32b5e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] Calling method '.version_select at 0x7f30a060cc10>' {{(pid=127143) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 04 08:57:44.901811 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: INFO nova.api.openstack.requestlog [None req-56a21f0c-ae24-40e1-b457-b84fa2c32b5e tempest-TestMinimumBasicScenario-1330805197 tempest-TestMinimumBasicScenario-1330805197-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-949961484" status: 202 len: 0 microversion: 2.1 time: 0.025274 Oct 04 08:57:44.902387 dv-cinder-860209-1-iscsi devstack@n-api.service[127143]: [pid: 127143|app: 0|req: 426/920] 10.222.0.16 () {58 vars in 1269 bytes} [Tue Oct 4 08:57:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-949961484 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)